Foreground Mitigation in Cosmological 21 cm Observations: From Component Separation to End-to-End Inference

Abstract

Redshifted 21 cm observations can trace neutral hydrogen from Cosmic Dawn and the Epoch of Reionization to lower-redshift large-scale structure, but foreground emission exceeds the cosmological fluctuations by several orders of magnitude. This review follows foreground mitigation from the measurement equation to final validation. We summarize spectral fitting, PCA/SVD, ICA, KL methods, GPR, GMCA, and DAYENU, including their signal loss in an idealized comparison. We then show how chromatic beams and frequency-dependent interferometric sampling mix angular foreground structure into line-of-sight modes and form the foreground wedge. Bayesian forward models and deep networks are assessed as routes to field and power-spectrum inference when the telescope response is included. Drawing on the SKA Science Data Challenge 3a, we summarize the experience gained and lessons learned, with emphasis on practical challenges in data processing, signal recovery, and uncertainty estimation. We conclude with field-level validation based on cross-correlation, transfer functions, and recovered power spectra.

1. Introduction

The redshifted 21 cm line provides a three-dimensional probe of neutral hydrogen from the Dark Ages and Cosmic Dawn through the Epoch of Reionization (EoR), and it traces large-scale structure after reionization. Its faint cosmological fluctuations are observed beneath Galactic and extragalactic radio emission that is several orders of magnitude brighter. Foreground mitigation is, therefore, required for both statistical detection and tomographic reconstruction, but it must preserve the cosmological modes used for inference [1,2,3,4].

This review examines how the assumptions behind foreground separation change once the frequency-dependent telescope response is included. It connects classical spectral and covariance methods to chromatic mode mixing, Bayesian forward models, and deep-learning reconstruction. An idealized numerical comparison isolates the signal loss caused by the separation algorithms, while the SKA Science Data Challenge 3a shows how these methods behave in a full interferometric pipeline. Particular attention is given to validation: foreground suppression alone is insufficient unless the recovery of the signal amplitude, the correlation between the reconstructed and true signal fields, recovered power, and uncertainty estimates are also tested. The main focus of this review is astrophysical foreground separation and chromatic mode mixing, while radio-frequency interference, calibration, ionospheric effects, and polarization leakage are mentioned only briefly in the context of their potential impact on signal recovery.

The structure of this paper is as follows. In Section 2, we describe the 21 cm signal and the physical basis of foreground spectral smoothness. In Section 3, we review statistical separation and filtering methods and compare them on an idealized data cube. In Section 4, we derive chromatic angular–spectral mode mixing and the foreground wedge in visibility and image space. In Section 5, we discuss joint Bayesian inference of the signal, foregrounds, and telescope response. In Section 6, we examine deep-learning methods for reconstruction. In Section 7, we summarize the lessons of the SKA Science Data Challenge 3a. In Section 8, we present field-level validation criteria and conclude with an outlook for future 21 cm analyses.

2. The 21 cm Signal and Foreground Emission

2.1. The 21 cm Signal as a Three-Dimensional Probe

The interval between recombination and the appearance of the first luminous sources remains one of the least directly observed stages of cosmic history. During the Dark Ages, the thermal evolution of the intergalactic medium (IGM) was governed primarily by cosmological expansion and atomic processes. The formation of the first stars, galaxies, and accreting black holes initiated Cosmic Dawn, coupling the spin temperature of neutral hydrogen to the gas through the Wouthuysen–Field effect and subsequently heating the IGM. Ultraviolet radiation from early galaxies then transformed the predominantly neutral medium into an ionized one during the Epoch of Reionization (EoR). These transitions encode information about the first sources, radiative feedback, structure formation, and potentially new physics that is difficult to recover from integrated observables alone [1,2,3,4].

The redshifted hyperfine line of neutral hydrogen provides a direct tracer of this evolution. Relative to the radio background, conventionally taken to be the cosmic microwave background (CMB), the differential brightness temperature may be written approximately as

$$\begin{array}{cc}\hfill \delta T_b(\mathit{x},z)& \simeq 27x_{\mathrm{HI}}(\mathit{x},z)\left[1+{\delta }_b(\mathit{x},z)\right]\left(1-{\displaystyle \frac{T_{\gamma }\left(z\right)}{T_s(\mathit{x},z)}}\right)\hfill \\ \hfill & \times \left({\displaystyle \frac{{\Omega }_b{h}^2}{0.023}}\right){\left[{\displaystyle \frac{0.15}{{\Omega }_m{h}^2}}{\displaystyle \frac{1+z}{10}}\right]}^{1/2}\left[{\displaystyle \frac{H\left(z\right)/(1+z)}{\mathrm{d}{v}_{\Vert }/\mathrm{d}{r}_{\Vert }}}\right]\mathrm{mK},\hfill \end{array}$$

(1)

where $x_{\mathrm{HI}}$ is the neutral fraction, ${\delta }_b$ is the baryon overdensity, $T_s$ is the spin temperature, $T_{\gamma }$ is the radio-background temperature, and the final factor accounts for the line-of-sight velocity gradient. Because observed frequency maps to redshift through $1+z={\nu }_{21}/{\nu }_{\mathrm{obs}}$, angular position and frequency jointly define a three-dimensional intensity field. The 21 cm signal can, therefore, trace not only the global timing of Cosmic Dawn and reionization but also their spatial morphology and scale dependence [1,4,5,6].

Figure 1 illustrates how different physical processes control the sign, amplitude, and spatial structure of the 21 cm signal. During the Dark Ages, collisions couple the spin temperature to the cooling gas and produce absorption against the CMB. As the density decreases, collisional coupling weakens and the signal moves toward zero. The first stars subsequently generate a Ly$\alpha$ radiation background, which couples the spin temperature back to the cold gas through the Wouthuysen–Field effect and produces a deeper, spatially inhomogeneous absorption signal. X-ray sources then heat the intergalactic medium, creating regions in absorption and emission before the mean signal changes sign once $T_s>T_{\gamma }$.

Figure 1. Schematic evolution of the redshifted 21 cm signal from $z\simeq 100$ to the present. The upper panels show illustrative brightness-temperature fields during collisional coupling, Ly$\alpha$ coupling, X-ray heating, and reionization. The lower panel shows a representative sky-averaged history, with circles marking the redshifts of the spatial slices. Negative and positive values denote absorption and emission relative to the CMB, respectively.

At later epochs, the heated neutral medium is observed in emission, while the growth of ionized bubbles drives $x_{\mathrm{HI}}$ and hence $\delta T_b$ toward zero inside those regions. This produces the strongly non-Gaussian and patchy morphology characteristic of reionization. After reionization, most of the diffuse intergalactic medium is ionized, but a small neutral component survives in self-shielded systems associated with galaxies. Its sub-mK integrated 21 cm emission is the signal targeted by post-reionization H I intensity-mapping surveys [7,8,9,10]. The upper panels emphasize that these transitions are spatially inhomogeneous, whereas the lower curve summarizes their approximate sky-averaged evolution. The amplitudes and transition redshifts shown here are schematic and depend on the astrophysical properties of the first radiation sources [1,6].

This capability motivates a broad experimental programme. Low-frequency interferometers including the Murchison Widefield Array (MWA), the Low-Frequency Array (LOFAR), and the Hydrogen Epoch of Reionization Array (HERA) have progressively improved upper limits on the EoR and Cosmic-Dawn power spectra [11,12,13,14,15,16]. SKA-Low is intended to combine substantially improved sensitivity with a dense core, broad frequency coverage, and imaging capability, enabling increasingly detailed statistical measurements and ultimately tomographic studies of the neutral IGM [17]. At lower redshift, 21 cm intensity mapping traces the collective emission of unresolved galaxies over large cosmological volumes and offers an independent route to measurements of large-scale structure, redshift-space distortions, and baryon acoustic oscillations [18,19,20]. Across these regimes, however, scientific return depends less on raw sensitivity alone than on whether foreground and instrumental uncertainties can be controlled without erasing or biasing the cosmological modes of interest.

2.2. Foreground Amplitudes, Spectral Smoothness, and Simulations

After calibration and imaging, an idealized frequency–angle data cube may be written as

$$y(\mathit{\theta },\nu )=s(\mathit{\theta },\nu )+f(\mathit{\theta },\nu )+n(\mathit{\theta },\nu ),$$

(2)

where $\mathit{\theta }=({\theta }_x,{\theta }_y)$ denotes the two-dimensional angular position on the sky, $\nu$ is the observing frequency, s is the cosmological 21 cm signal, f is the total astrophysical foreground, and  n denotes thermal noise. Across the frequency ranges used for Cosmic-Dawn, EoR, and post-reionization intensity mapping, foreground brightness temperatures are commonly several orders of magnitude larger than the cosmological fluctuations. Galactic synchrotron emission is normally the dominant component, with additional contributions from Galactic free–free emission, diffuse and compact extragalactic radio sources, and polarized emission that can leak into total intensity [4,21,22]. Foreground subtraction is nevertheless possible because continuum emission is generally much more coherent in frequency than the redshifted line signal.

The spectral smoothness has a physical origin. Synchrotron radiation from an approximately power-law distribution of relativistic electrons is locally close to a power law in frequency. Free–free emission is also smooth, while the spectra of most radio sources vary gradually over a moderate fractional bandwidth. Frequently used phenomenological models are

$$\begin{array}{cc}\hfill T_{\mathrm{syn}}(\mathit{\theta },\nu )& =A_{\mathrm{syn}}\left(\mathit{\theta }\right){\left({\displaystyle \frac{\nu }{{\nu }_0}}\right)}^{-{\beta }_{\mathrm{syn}}\left(\mathit{\theta }\right)-c_{\mathrm{syn}}\left(\mathit{\theta }\right)ln(\nu /{\nu }_0)},\hfill \\ \hfill T_{\mathrm{ff}}(\mathit{\theta },\nu )& \simeq A_{\mathrm{ff}}\left(\mathit{\theta }\right){\left({\displaystyle \frac{\nu }{{\nu }_0}}\right)}^{-2.1},I_{\mathrm{ps}}(\mathit{\theta },\nu )=\sum _i{S}_{i0}{\left({\displaystyle \frac{\nu }{{\nu }_0}}\right)}^{-{\alpha }_i}{\delta }_{\mathrm{D}}^{\left(2\right)}(\mathit{\theta }-{\mathit{\theta }}_i).\hfill \end{array}$$

(3)

Here ${\beta }_{\mathrm{syn}}$ is the synchrotron brightness-temperature spectral index, $c_{\mathrm{syn}}$ describes slow spectral curvature, and ${\alpha }_i$ is the spectral index of the ith compact source. Spatial variations of these parameters mean that the foreground cube is not described by one universal power law. Instead, it is usually well approximated by a small set of smooth spectral degrees of freedom. Low-frequency survey maps and global sky models are widely used to constrain the amplitudes and spatial variation of these components [23,24,25,26,27].

This smoothness is not exact. Different lines of sight contain different mixtures of synchrotron-emitting regions, and individual radio sources have a distribution of spectral indices and curvatures. In polarization, $Q+iU\propto exp\left(2i\varphi {\lambda }^2\right)$ in a simple Faraday-thin model, so even a small frequency-dependent leakage into Stokes I can produce rapidly varying structure [22]. These effects increase the number of foreground modes required by a subtraction algorithm. Instrumental chromaticity causes an additional and more serious loss of smoothness, but it is deliberately excluded from the idealized demonstration below and considered separately in the following section.

Several complementary foreground-simulation strategies are used in the literature. In empirical-template simulations, a reference map such as the 408-MHz survey is extrapolated with either a fixed spectral index or a spatially varying index and curvature. Small-scale Gaussian realizations may be added when the angular resolution of the template is insufficient. Global-sky-model simulations instead combine several radio surveys to predict the sky at arbitrary frequencies [23,24,25,26]. These approaches reproduce recognizable sky morphology, although their frequency coherence is inherited from the adopted spectral model.

A second class of simulations specifies the multifrequency angular covariance of each foreground component. A commonly used form is

$$C_{\mathit{\ell }}^{\left(a\right)}(\nu ,{\nu }^{\prime })=A_a{\left({\displaystyle \frac{\mathit{\ell }}{{\mathit{\ell }}_0}}\right)}^{-{\gamma }_a}{\left({\displaystyle \frac{\nu {\nu }^{\prime }}{{\nu }_0^2}}\right)}^{-{\beta }_a}exp\left[-{\displaystyle \frac{{ln}^2(\nu /{\nu }^{\prime })}{2{\xi }_a^2}}\right],$$

(4)

where a labels the foreground component. The parameters $A_a$, ${\gamma }_a$, and ${\beta }_a$ control its amplitude, angular power-law slope, and  mean spectral index, while ${\xi }_a$ sets the frequency-coherence length. This construction is convenient for producing large ensembles and for testing how cleaning changes as the foreground coherence is varied [28].

Compact-source simulations usually draw source fluxes from $\mathrm{d}N/\mathrm{d}S$, assign Poisson-distributed or clustered positions, and sample spectral indices and curvatures from empirical distributions. Sources above a specified flux threshold can be removed before the unresolved population is added to the diffuse foreground. More complete end-to-end sky models combine diffuse synchrotron and free–free emission, compact sources, polarization and Faraday rotation, and independent 21 cm realizations. The LOFAR-EoR simulations developed by Jelić et al. [21,22] are influential examples. Comparative intensity-mapping studies and blind foreground challenges have subsequently tested multiple sky prescriptions and cleaning pipelines on common simulated data sets [29,30]. No single simulation is sufficient: template-based skies provide realistic morphology, covariance-based skies provide controlled ensembles, source catalogs capture non-Gaussian compact structure, and end-to-end simulations test interactions among components.

3. Statistical Foreground Separation

Foreground-separation methods distinguish components through spectral shape, empirical rank, statistical independence, covariance, sparsity, or delay-domain localization. The assumptions determine both the residual contamination and which cosmological modes are attenuated. The following subsections first summarize the main methods and then compare them on a common idealized data cube.

3.1. Parametric Spectral Fitting

The most direct subtraction method fits a smooth function independently to each line of sight. A common choice is a polynomial in log-frequency,

$$ln\widehat{f}(\mathit{\theta },\nu )={\displaystyle \sum _{q=0}^p}{a}_q\left(\mathit{\theta }\right){\left[ln\left({\displaystyle \frac{\nu }{{\nu }_0}}\right)\right]}^q,\widehat{s}=y-\widehat{f}.$$

(5)

Low polynomial order protects the cosmological signal but may leave spectral curvature unmodeled; high order reduces the foreground residual while removing slowly varying 21 cm modes. Weighted least squares can accommodate heteroscedastic noise, but missing frequency channels make the fitted basis less orthogonal and can increase spectral leakage. Polynomial fitting is transparent and computationally inexpensive, although it cannot easily represent spatially varying mixtures whose spectra are not captured by the chosen basis [29,31].

3.2. PCA and SVD

For PCA and SVD, the cube is reshaped into $\mathit{X}\in {\mathbb{R}}^{N_{\nu }\times N_p}$, with frequency channels as rows and angular pixels as columns. After subtracting the appropriate mean and, when needed, applying noise weights, one computes

$$\mathit{X}=\mathit{U}\Sigma {\mathit{V}}^{\mathsf{T}},{\widehat{\mathit{F}}}_{\mathrm{PCA}}={\mathit{U}}_m{\mathit{U}}_m^{\mathsf{T}}\mathit{X},{\widehat{\mathit{S}}}_{\mathrm{PCA}}=\left(\mathit{I}-{\mathit{U}}_m{\mathit{U}}_m^{\mathsf{T}}\right)\mathit{X},$$

(6)

where ${\mathit{U}}_m$ contains the first m left singular vectors. These vectors are also the leading eigenvectors of the empirical frequency covariance ${\widehat{\mathit{C}}}_X=\mathit{X}{\mathit{X}}^{\mathsf{T}}/N_p$, so covariance PCA and SVD implement the same spectral projection. Because the foreground dominates the variance, its smooth spectral modes usually appear among the largest singular values. The principal choice is the number m of removed modes. Removing too few modes leaves foreground power; removing too many suppresses the cosmological signal, especially at low $k_{\Vert }$. Eigenvalue inspection is useful for selecting m, but signal injection and recovery are required to measure the resulting transfer function [29].

Several extensions have been proposed to reduce the signal loss associated with standard PCA. Zuo et al. [32] applied robust principal component analysis to decompose the frequency covariance into a low-rank foreground component and a sparse 21 cm component, thereby avoiding a simple hard truncation of the leading eigenmodes. More recently, Zuo et al. [33] demonstrated the mathematical equivalence of PCA and SVD foreground subtraction and proposed a semiblind singular-vector projection method. By incorporating prior information about the foreground singular subspace without requiring its absolute amplitude, this approach can improve foreground removal while reducing cosmological signal loss. Both methods nevertheless rely on assumptions about the foreground or signal subspaces, and their transfer functions must be validated using simulations or signal-injection tests.

3.3. Independent Component Analysis

ICA treats the foreground as a linear mixture of statistically independent latent components rather than orthogonal variance modes. After centering and whitening the data, FastICA searches for a demixing matrix $\mathit{W}$ such that $\mathit{Z}=\mathit{W}\mathit{X}$ has maximally non-Gaussian and approximately independent rows. A common contrast is an approximation to negentropy,

$$\widehat{\mathit{W}}=arg\underset{\mathit{W}{\mathit{W}}^{\mathsf{T}}=\mathit{I}}{max}{\displaystyle \sum _{j=1}^m}{\left[\mathbb{E}\left\{G\left(z_j\right)\right\}-\mathbb{E}\left\{G\left(g\right)\right\}\right]}^2,{\widehat{\mathit{F}}}_{\mathrm{ICA}}=\widehat{\mathit{A}}\widehat{\mathit{Z}},$$

(7)

where g is a standardized Gaussian variable, $\widehat{\mathit{A}}={\widehat{\mathit{W}}}^{+}$, and G is a non-quadratic contrast function. ICA uses higher-order statistics and does not require its recovered components to be orthogonal. The component ordering, normalization, and  physical labels are not uniquely determined, but foreground subtraction only requires an accurate reconstruction of their sum. If the relevant information is nearly Gaussian or dominated by second-order covariance, ICA often approaches a rotation of the PCA foreground subspace, explaining their similar performance in many idealized tests [29,34,35,36].

3.4. Karhunen–Lo Ève Transform

The KL transform uses prior covariance models to construct modes ordered by their expected signal-to-contaminant ratio. Let ${\mathit{C}}_C={\mathit{C}}_F+{\mathit{C}}_N$ and factor ${\mathit{C}}_C=\mathit{L}{\mathit{L}}^{\mathsf{T}}$. The whitened signal covariance and KL coordinates are

$${\mathit{L}}^{-1}{\mathit{C}}_S{\mathit{L}}^{-\mathsf{T}}=\mathit{U}\mathbf{\Lambda }{\mathit{U}}^{\mathsf{T}},\mathit{z}={\mathit{U}}^{\mathsf{T}}{\mathit{L}}^{-1}\mathit{x},{\widehat{\mathit{s}}}_{\mathrm{KL}}=\mathit{L}\mathit{U}\mathit{D}{\mathit{U}}^{\mathsf{T}}{\mathit{L}}^{-1}\mathit{x},$$

(8)

where $\mathbf{\Lambda }=diag\left({\lambda }_i\right)$ contains the expected signal-to-contaminant variance ratios. The selection matrix $\mathit{D}$ is diagonal, with $D_{ii}=1$ for retained modes and $D_{ii}=0$ for discarded modes. The KL operation is, therefore, a covariance-defined change of basis followed by explicit mode selection. It can retain a low-total-variance mode when that mode is expected to have high signal purity, which is not possible with variance ordering alone. Its weakness is dependence on the assumed ${\mathit{C}}_S$, ${\mathit{C}}_F$, ${\mathit{C}}_N$, and instrumental response. A  mis-specified covariance rotates the retained subspace and can either leave foreground residuals or reject cosmological modes. KL filtering is especially convenient in bases where symmetries render the covariance block diagonal, such as the m-mode representation of transit interferometers [18,37].

A practical limitation of a global KL transform is that a single covariance model may not adequately describe foreground statistics that vary with angular scale and sky position. Needlet Karhunen–Loève (NKL) cleaning addresses this issue by performing the KL decomposition in localized needlet domains, allowing the signal and foreground covariances to vary across both scale and position [38]. A related but conceptually distinct needlet-space approach is the generalized needlet internal linear combination (GNILC), which uses locally estimated covariance information to identify the contaminant subspace and construct a minimum-variance estimate of the H i signal, rather than ordering modes according to the KL signal-to-foreground criterion [39].

3.5. Gaussian-Process Regression

At each angular pixel, GPR models the frequency dependence of the data components as stochastic functions. Let $\mathit{x}=\mathit{f}+\mathit{s}+\mathit{n}$ denote the data vector sampled over the set of observing-frequency channels. We assign independent covariance matrices ${\mathit{K}}_F$, ${\mathit{K}}_S$, and ${\mathit{K}}_N$ to the foreground, 21 cm signal, and noise components, respectively, so that $\mathit{K}={\mathit{K}}_F+{\mathit{K}}_S+{\mathit{K}}_N$. Gaussian conditioning gives

$$\begin{array}{cc}\mathbb{E}[\mathit{f}\mid \mathit{x}]={\mathit{K}}_F{\mathit{K}}^{-1}\mathit{x},\hfill & \hfill Cov(\mathit{f}\mid \mathit{x})={\mathit{K}}_F-{\mathit{K}}_F{\mathit{K}}^{-1}{\mathit{K}}_F,\\ \hfill lnp(\mathit{x}\mid \mathbf{\Theta })=-{\displaystyle \frac{1}{2}}{\mathit{x}}^{\mathsf{T}}{\mathit{K}}^{-1}\mathit{x}-{\displaystyle \frac{1}{2}}ln\left|\mathit{K}\right|-{\displaystyle \frac{N_{\nu }}{2}}ln\left(2\pi \right).& \hfill \end{array}$$

(9)

Squared-exponential or long-coherence Matérn kernels are commonly used for the smooth foreground, while shorter-coherence Matérn or exponential kernels can represent the 21 cm signal and other residual components. White thermal noise contributes a diagonal kernel. Kernel amplitudes and coherence lengths may be optimized with the marginal likelihood or marginalized over. GPR provides a foreground posterior and an uncertainty conditional on the adopted kernels, rather than only a best-fitting low-rank subspace. Its central difficulty is identifiability: foreground, signal, and instrumental-residual kernels can overlap, and a mis-specified signal covariance can bias the recovered power spectrum [40,41,42].

3.6. Generalized Morphological Component Analysis

GMCA distinguishes components through sparsity and morphological diversity. It models the foreground as $\mathit{F}=\mathit{A}\mathit{Z}$ and assumes that the source maps have sparse coefficients $\mathit{\alpha }$ in a dictionary $\mathbf{\Phi }$, usually a multiscale wavelet or starlet transform. A schematic objective is

$$(\widehat{\mathit{A}},\widehat{\mathit{\alpha }})=arg\underset{\mathit{A},\mathit{\alpha }}{min}{\displaystyle \frac{1}{2}}{∥\mathit{X}-\mathit{A}\mathit{\alpha }\mathbf{\Phi }∥}_F^2+{\displaystyle \sum _{j=1}^m}{\lambda }_j{\parallel {\mathit{\alpha }}_j\parallel }_1,{\widehat{\mathit{F}}}_{\mathrm{GMCA}}=\widehat{\mathit{A}}\widehat{\mathit{\alpha }}\mathbf{\Phi }.$$

(10)

The algorithm alternates between estimating the spectral mixing matrix and thresholding transform coefficients. It can exploit angular morphology that is ignored by independent line-of-sight fitting and is well suited to non-Gaussian diffuse and compact foregrounds. Its performance depends on the chosen dictionary, the number of sources, and the threshold schedule. Cosmological structures that are sparse in the same dictionary can be absorbed into the foreground reconstruction [43,44,45].

3.7. DAYENU Spectral Filtering

DAYENU is a linear high-dynamic-range filter designed primarily for intensity-mapping power-spectrum estimation. Instead of fitting detailed foreground amplitudes or a sky-dependent mixing matrix, it models smooth foregrounds as being confined within a delay range $\left|\tau \right|\lesssim {\tau }_{\mathrm{cut}}$. For regularly sampled frequencies, its foreground-plus-identity covariance model and filter may be written as

$${\left[{\mathit{C}}_{\mathrm{D}}\right]}_{ij}={ϵ}^{-1}sinc\left[2\pi {\tau }_{\mathrm{cut}}({\nu }_i-{\nu }_j)\right]+{\delta }_{ij},{\mathit{R}}_{\mathrm{D}}={\mathit{C}}_{\mathrm{D}}^{-1},{\mathit{x}}_{\mathrm{D}}={\mathit{R}}_{\mathrm{D}}\mathit{x},$$

(11)

where $sinc\left(x\right)=sin\left(x\right)/x$ and $ϵ$ specifies the desired suppression of the smooth subspace. With flagged channels, the corresponding masked covariance is filtered using a Moore–Penrose pseudoinverse. The  eigenvectors of the sinc covariance are discrete prolate spheroidal sequences (DPSSs), which optimally concentrate spectral power within a finite delay interval. This allows DAYENU to suppress smooth foreground modes without constructing a detailed empirical covariance from the data, and it behaves more regularly than a direct tapered Fourier transform in the presence of missing channels.

The parameters ${\tau }_{\mathrm{cut}}$ and $ϵ$ remain important: increasing the delay range removes more contamination but also suppresses additional low-$k_{\Vert }$ cosmological information. The filtered cube should, therefore, be regarded as an input to a properly normalized power-spectrum estimator, not automatically as an unbiased reconstruction of the 21 cm sky [46].

Table 1 summarizes the main assumptions, operations, and limitations of the foreground-mitigation methods discussed above. The methods differ not only in their numerical implementation, but also in the information used to identify foreground-contaminated modes. Parametric fitting imposes an explicit smooth spectral form; PCA/SVD and ICA infer foreground components directly from the data; KL and GPR require covariance information; GMCA exploits sparsity in a transformed spatial domain; and DAYENU suppresses a predefined low-delay spectral subspace.

Table 1. Summary of foreground-separation and filtering methods.

Method	Assumption	Operation	Main Limitation
Parametric fitting	Smooth spectral function	Fits and subtracts a low-order polynomial	Sensitive to polynomial order
PCA/SVD	Low-rank, high-variance foregrounds	Removes leading spectral eigenmodes	Component choice and signal loss
ICA	Independent non-Gaussian components	Estimates mixing and demixing matrices	Weak separation of correlated components
KL transform	Known signal and contaminant covariances	Retains high-purity covariance eigenmodes	Sensitive to covariance mismatch
GPR	Distinct covariance kernels	Estimates the foreground posterior	Kernel degeneracy and mis-specification
GMCA	Sparse, morphologically distinct components	Performs sparse component separation	Dictionary and regularization dependence
DAYENU	Foregrounds confined to low delays	Filters the low-delay DPSS subspace	Low-$k_{\Vert }$ signal attenuation

The comparison in Table 1 also clarifies that these algorithms do not produce equivalent data products. PCA/SVD, ICA, and GMCA primarily estimate an empirical foreground model and subtract it from the data. KL performs covariance-informed mode selection, while GPR returns a kernel-dependent posterior foreground estimate. DAYENU is more naturally interpreted as a spectral filter for subsequent power-spectrum estimation rather than as a direct reconstruction of the 21 cm brightness-temperature field. Consequently, the visual quality of a cleaned map cannot by itself be used to rank the methods. Their performance must instead be evaluated through foreground residuals, signal transfer functions, recovered power spectra, and  uncertainty calibration.

3.8. Idealized Comparison and Signal Loss

We compare the component-separation methods using the same idealized frequency–angular data cube. The input contains a cosmological 21 cm signal s, smooth foreground emission f, and Gaussian thermal noise n. The  foreground is represented by three spectral components corresponding approximately to diffuse synchrotron emission, free–free emission, and an unresolved-source population. No primary-beam chromaticity, frequency-dependent PSF, calibration residual, flagging, or other interferometric effect is included. The experiment, therefore, isolates the intrinsic separation and signal-loss behaviour of the algorithms.

The 21 cm field is correlated between neighbouring frequency channels and has an rms amplitude close to $5\mathrm{mK}$ across the band, with only weak frequency evolution. The foreground rms is approximately $50\mathrm{K}$, while the thermal-noise rms is $0.5\mathrm{mK}$. The foreground is, therefore, about four orders of magnitude brighter than the cosmological signal. The  principal simulation parameters are listed in Table 2.

Table 2. Parameters of the idealized simulation.

Parameter	Adopted Value or Model
Frequency coverage	120–$180\mathrm{MHz}$ with 64 channels
Angular grid	$44\times 44$ pixels, corresponding to 1936 lines of sight
Random realization	Fixed random seed of 19
Foreground model	Three smooth spectral components with diffuse and compact angular structure
Foreground amplitude	Rms fluctuation of $50\mathrm{K}$
21 cm amplitude	Rms fluctuation of approximately $5\mathrm{mK}$, varying from about $5.17$ to $4.87\mathrm{mK}$ across the band
21 cm frequency structure	Exponential correlation with a characteristic scale of $1.8\mathrm{MHz}$
21 cm angular structure	Correlated random fields smoothed on a scale of approximately $0.75$ pixels
Thermal noise	Independent Gaussian noise with an rms of $0.5\mathrm{mK}$
Brightness-temperature reference	$6.89\lesssim z\lesssim 10.84$, saturated heating $T_s\gg T_{\gamma }$, and a unit velocity-gradient factor

All methods are applied to the same simulated cube. The foreground rank is fixed to three and is supplied to the methods that require a component number. This is a favourable case: choosing too few components would leave foreground residuals, whereas choosing too many would remove additional 21 cm modes. The adopted algorithmic parameters and the resulting reconstruction metrics are summarized in Table 3. Here, r is the Pearson correlation coefficient between the reconstructed and true 21 cm fields. The normalized root-mean-square error (nRMSE) is defined as

$$\mathrm{nRMSE}={\displaystyle \frac{\sqrt{{⟨{\left[\widehat{s}(\mathit{\theta },\nu )-s(\mathit{\theta },\nu )\right]}^2⟩}_{\mathit{\theta },\nu }}}{{\sigma }_s}},{\sigma }_s=\sqrt{{⟨{\left[s(\mathit{\theta },\nu )-⟨s⟩\right]}^2⟩}_{\mathit{\theta },\nu }}.$$

(12)

Here the averages are taken over all angular pixels and frequency channels. The nRMSE is dimensionless, with $\mathrm{nRMSE}=0$ corresponding to an exact reconstruction. For example, $\mathrm{nRMSE}=0.4$ means that the rms reconstruction error is $40\%$ of the rms amplitude of the true 21 cm field.

Table 3. Algorithms, adopted parameters, and reconstruction accuracy.

Method	Adopted Configuration	r	nRMSE
Parametric fitting	Fifth-order Legendre polynomial fitted independently to each line of sight	$0.818$	$0.578$
PCA/SVD	Removal of the first three singular modes	$0.912$	$0.411$
FastICA	Three independent components, unit-variance whitening, log-cosh contrast, tolerance ${10}^{-3}$, and at most 5000 iterations	$0.912$	$0.411$
KL transform	Simulation-derived signal and contaminant covariances, with three lowest-purity modes rejected	$0.924$	$0.381$
GPR	Foreground correlation scales of 28 and $12\mathrm{MHz}$, a 21 cm correlation scale of $1.8\mathrm{MHz}$, and a noise rms of $0.5\mathrm{mK}$	$0.912$	$0.411$
GMCA-style	Three foreground components, a two-dimensional discrete-cosine dictionary, and 45 iterations	$0.911$	$0.412$
DAYENU	Delay cutoff of $4\mathrm{ns}$ and suppression parameter $ϵ=3\times {10}^{-13}$	$0.822$	$0.570$

The KL and GPR results should be interpreted with some care. Both methods use covariance information derived from the simulation, whereas PCA/SVD, FastICA, and GMCA estimate the foreground subspace primarily from the data. The KL result is consequently a supervised best-case example rather than evidence that the method is universally more accurate.

Figure 2 shows a representative channel at $\nu \simeq 150.5\mathrm{MHz}$, corresponding to $z\simeq 8.44$. The input map is visually almost identical to the foreground map because the cosmological signal and thermal noise are negligible on the kelvin colour scale. The  component-amplitude panel makes the dynamic range clearer: the foreground remains at tens of kelvin, the 21 cm signal stays close to $5\mathrm{mK}$, and the noise remains near $0.5\mathrm{mK}$.

Figure 2. Idealized foreground separation at $150.5\mathrm{MHz}$. The first row shows the input, foreground, true 21 cm signal, and noise; the remaining panels show component amplitudes and recovered signals.

The reconstructed maps recover much of the morphology of the true 21 cm field. PCA/SVD, FastICA, GPR, and the GMCA-style method give similar results because the simulated foreground is exactly smooth and low rank. Parametric fitting produces a visibly more attenuated reconstruction because some spectrally smooth 21 cm structure is absorbed into the polynomial foreground model. The KL reconstruction is closest to the true map in this realization, partly because the simulation supplies the covariance information required to order the modes.

The left panel of Figure 3 compares the true and reconstructed signals along a representative line of sight. All methods recover the main channel-to-channel variations, but none reproduces the true signal exactly. The differences are dominated by foreground-induced signal loss rather than thermal noise, since the adopted noise amplitude is well below the 21 cm rms.

Figure 3. (a) Recovered 21 cm signals along representative lines of sight; (b) normalized frequency-mode power spectra; and (c) spherically averaged three-dimensional power spectra.

The middle panel shows the mean power after Fourier transformation along the frequency direction. The curves are normalized by the maximum non-zero-mode power of the true 21 cm cube. The largest deviations occur at the largest spectral scales, where the 21 cm signal varies slowly and overlaps most strongly with the foreground spectral subspace. Intermediate and high frequency modes are recovered more accurately because they are more distinct from smooth continuum emission.

The right panel shows the ratio of the recovered to true 21 cm power spectra. Most methods approach unity over the better-recovered scales, whereas the recovered power tends to be lower at small k, where the slowly varying 21 cm modes overlap most strongly with the smooth foreground components and are, therefore, more susceptible to signal loss. The Hanning window is applied before the Fourier transform to reduce boundary discontinuities and spectral leakage. A ratio close to unity does not necessarily imply accurate signal recovery: residual foregrounds can increase the recovered power, whereas foreground subtraction can remove part of the 21 cm signal and reduce it. These two effects may partially cancel in the power-spectrum ratio, even when the recovered field itself differs from the true 21 cm signal.

This behaviour illustrates the fundamental trade-off in foreground subtraction. A more aggressive foreground model can reduce residual contamination, but it also removes a larger fraction of the cosmological signal. The problem remains present even in this instrument-free simulation: spectrally smooth 21 cm modes cannot be distinguished perfectly from smooth foreground modes using frequency structure alone.

DAYENU should be interpreted as a spectral filter rather than as a direct, unbiased reconstruction of the sky. In this comparison, the final DAYENU output is evaluated directly against the unfiltered true s. For  the adopted parameters, it gives a correlation coefficient of approximately $0.822$ and an nRMSE of approximately $0.570$, as reported in Table 3. Its lower reconstruction accuracy mainly reflects the attenuation of spectrally smooth 21 cm modes that overlap with the foreground delay range. The difference between the filtered output and s may also contain residual foreground and filtered thermal noise, so it should not be interpreted as pure signal loss alone.

The DAYENU parameters were examined over delay cutoffs of 2–$15\mathrm{ns}$ and suppression parameters between ${10}^{-14}$ and ${10}^{-8}$. In this idealized demonstration, the preferred parameters were identified by comparing the filtered result with the known simulated 21 cm field. Such truth-assisted parameter selection is not available for real observations. In practical analyses, the signal attenuation and power-spectrum transfer function must instead be calibrated using signal injections or end-to-end simulations.

Overall, the comparison represents an optimistic separation limit. The  foreground is exactly smooth, its rank is known, and the instrumental response is frequency independent. Primary-beam chromaticity, frequency-dependent sampling, calibration residuals, flagged channels, polarization leakage, and  other instrumental effects would introduce additional spectral structure and increase both foreground leakage and signal loss. The performance shown here should, therefore, be interpreted as a controlled demonstration of the algorithms rather than a forecast for a specific 21 cm experiment.

4. Instrumental Chromaticity and Foreground Mode Mixing

The additive sky model is incomplete for interferometric data because the instrumental response is chromatic, i.e., it varies with observing frequency (or wavelength). In particular, the baseline sampling, primary beam, and  synthesized beam are frequency dependent. This chromatic response transfers angular foreground structure into spectral modes, producing foreground mode mixing. This section connects the same effect in visibility, delay, and image space.

4.1. Chromatic Visibility Sampling and the Foreground Wedge

For a projected physical baseline $\mathit{b}$, the narrow-field visibility at frequency $\nu$ is

$$V(\mathit{b},\nu )=\int {\mathrm{d}}^2\mathit{\theta }A(\mathit{\theta },\nu )I(\mathit{\theta },\nu )exp\left[-2\pi i{\displaystyle \frac{\nu }{c}}\mathit{b}·\mathit{\theta }\right],$$

(13)

where $\mathit{\theta }$ is the angular position relative to the phase centre, $A(\mathit{\theta },\nu )$ is the primary beam, $I(\mathit{\theta },\nu )$ is the sky brightness, and c is the speed of light. The angular Fourier coordinate sampled by the baseline is $\mathit{u}\left(\nu \right)=\nu \mathit{b}/c$. A fixed physical baseline, therefore, moves through the $uv$ plane as the observing frequency changes. Consequently, different angular Fourier modes of the foreground are sampled in different frequency channels, even when the intrinsic foreground spectrum is smooth.

This angular–spectral coupling can be derived directly from the visibility equation. Define the angular Fourier transform of the beam-weighted sky by

$$\widetilde{\left(AI\right)}(\mathit{u},\nu )\equiv \int {\mathrm{d}}^2\mathit{\theta }A(\mathit{\theta },\nu )I(\mathit{\theta },\nu )e^{-2\pi i\mathit{u}·\mathit{\theta }}.$$

(14)

Equation (13) can then be written as $V(\mathit{b},\nu )=\widetilde{\left(AI\right)}[\mathit{u}\left(\nu \right),\nu ]$. Because the visibility depends on frequency both explicitly and through the sampled coordinate $\mathit{u}\left(\nu \right)$, the chain rule gives

$$\begin{array}{cc}\hfill {\displaystyle \frac{\mathrm{d}V(\mathit{b},\nu )}{\mathrm{d}\nu }}& ={\left.{\displaystyle \frac{\partial \widetilde{\left(AI\right)}(\mathit{u},\nu )}{\partial \nu }}\right|}_{\mathit{u}}+{\displaystyle \frac{\mathrm{d}\mathit{u}}{\mathrm{d}\nu }}·{\nabla }_{\mathit{u}}\widetilde{\left(AI\right)}(\mathit{u},\nu )\hfill \\ \hfill & ={\left.{\displaystyle \frac{\partial \widetilde{\left(AI\right)}(\mathit{u},\nu )}{\partial \nu }}\right|}_{\mathit{u}}+{\displaystyle \frac{\mathit{b}}{c}}·{\nabla }_{\mathit{u}}\widetilde{\left(AI\right)}(\mathit{u},\nu ),\mathit{u}={\displaystyle \frac{\nu }{c}}\mathit{b}.\hfill \end{array}$$

(15)

The first term describes the explicit frequency dependence of the beam-weighted sky at a fixed angular Fourier coordinate. It includes both the intrinsic foreground spectrum and any primary-beam chromaticity. The second term arises because the baseline moves across the angular Fourier plane as the frequency changes. It converts spatial gradients in the $uv$ plane into spectral variation and is, therefore, the baseline-migration contribution to mode mixing.

To isolate this second contribution, consider an achromatic primary beam, $A(\mathit{\theta },\nu )=A_0\left(\mathit{\theta }\right)$, and a separable foreground, $I(\mathit{\theta },\nu )=I_0\left(\mathit{\theta }\right)f\left(\nu \right)$. The frequency derivative then becomes

$${\displaystyle \frac{\mathrm{d}{V}_{\mathrm{fg}}}{\mathrm{d}\nu }}=\underset{\mathrm{intrinsic}\mathrm{spectral}\mathrm{variation}}{\underbrace{f^{\prime }\left(\nu \right)\widetilde{\left(A_0{I}_0\right)}\left(\mathit{u}\right)}}+\underset{\mathrm{mode}\mathrm{mixing}}{\underbrace{f\left(\nu \right){\displaystyle \frac{\mathit{b}}{c}}·{\nabla }_{\mathit{u}}\widetilde{\left(A_0{I}_0\right)}\left(\mathit{u}\right)}},\mathit{u}={\displaystyle \frac{\nu }{c}}\mathit{b}.$$

(16)

The second term is the essential mode-mixing term. It depends on the baseline and on the angular Fourier structure of the foreground, rather than on the intrinsic spectral derivative $f^{\prime }\left(\nu \right)$. In particular, for a perfectly flat foreground spectrum, $f\left(\nu \right)\propto {\nu }^0$ and hence $f^{\prime }\left(\nu \right)=0$, but the measured visibility generally remains frequency dependent because ${\nabla }_{\mathit{u}}\widetilde{\left(A_0{I}_0\right)}$ is non-zero. An intrinsically frequency-independent foreground can, therefore, acquire spectral structure solely through chromatic interferometric sampling.

The same effect can be viewed in image space through the interferometric point-spread function. Let $\mathcal{S}(\mathit{u},\nu )$ denote the visibility sampling function. For $N_s$ discrete Fourier samples at ${\mathit{u}}_j\left(\nu \right)=\nu {\mathit{b}}_j/c$, an idealized unweighted sampling function and its corresponding point-spread function are

$$\mathcal{S}(\mathit{u},\nu )={\displaystyle \frac{1}{N_s}}{\displaystyle \sum _{j=1}^{N_s}}{\delta }_{\mathrm{D}}^{\left(2\right)}\left[\mathit{u}-{\mathit{u}}_j\left(\nu \right)\right],\mathrm{PSF}(\mathit{\theta },\nu )\equiv {\mathcal{F}}_{\mathit{u}\to \mathit{\theta }}^{-1}\left\{\mathcal{S}(\mathit{u},\nu )\right\}={\displaystyle \frac{1}{N_s}}{\displaystyle \sum _{j=1}^{N_s}}exp\left(2\pi i{\displaystyle \frac{\nu }{c}}{\mathit{b}}_j·\mathit{\theta }\right).$$

(17)

Here, ${\delta }_{\mathrm{D}}^{\left(2\right)}$ is the two-dimensional Dirac delta function, ${\mathcal{F}}_{\mathit{u}\to \mathit{\theta }}^{-1}$ denotes the inverse two-dimensional Fourier transform, and the normalization is chosen so that $\mathrm{PSF}(\mathit{0},\nu )=1$. The sampled coordinates ${\mathit{u}}_j\left(\nu \right)$ change with frequency, so the synthesized $\mathrm{PSF}(\mathit{\theta },\nu )$ is chromatic even though the physical array does not change.

A flat-spectrum point source provides a simple illustration. Consider a source at ${\mathit{\theta }}_s$ with $I_{\mathrm{sky}}(\mathit{\theta },\nu )=I_s{\delta }_{\mathrm{D}}^{\left(2\right)}(\mathit{\theta }-{\mathit{\theta }}_s)$, where the constant amplitude $I_s$ is proportional to ${\nu }^0$. We neglect primary-beam chromaticity so that all measured frequency structure is produced by the interferometric response. Suppose that the source is observed over a top-hat band centred at ${\nu }_0$ with bandwidth $\Delta \nu$, such that ${\nu }_0-\Delta \nu /2\le \nu \le {\nu }_0+\Delta \nu /2$. The dirty image $I_{\mathrm{D}}$ and its delay transform are then

$$\begin{array}{cc}\hfill I_{\mathrm{D}}(\mathit{\theta },\nu )& =I_s\mathrm{PSF}(\mathit{\theta }-{\mathit{\theta }}_s,\nu )\hfill \\ \hfill & ={\displaystyle \frac{I_s}{N_s}}{\displaystyle \sum _{j=1}^{N_s}}exp\left(2\pi i\nu {\tau }_j\right),{\tau }_j\equiv {\displaystyle \frac{{\mathit{b}}_j·(\mathit{\theta }-{\mathit{\theta }}_s)}{c}},\hfill \\ \hfill {\tilde{I}}_{\mathrm{D}}(\mathit{\theta },\tau )& \equiv {\int }_{{\nu }_0-\Delta \nu /2}^{{\nu }_0+\Delta \nu /2}\mathrm{d}\nu I_{\mathrm{D}}(\mathit{\theta },\nu )e^{-2\pi i\nu \tau }={\displaystyle \frac{I_s}{N_s}}{\displaystyle \sum _{j=1}^{N_s}}{\int }_{{\nu }_0-\Delta \nu /2}^{{\nu }_0+\Delta \nu /2}\mathrm{d}\nu e^{-2\pi i\nu (\tau -{\tau }_j)}\hfill \\ \hfill & ={\displaystyle \frac{I_s\Delta \nu }{N_s}}{\displaystyle \sum _{j=1}^{N_s}}{e}^{-2\pi i{\nu }_0(\tau -{\tau }_j)}sinc\left[\pi \Delta \nu (\tau -{\tau }_j)\right],sinc\left(x\right)\equiv {\displaystyle \frac{sinx}{x}}.\hfill \end{array}$$

(18)

Here, $\tau$ is the Fourier variable conjugate to frequency, and ${\tau }_j$ is the geometric delay associated with the jth baseline and the angular separation between the image position and the source.

Equation (18) shows mode mixing directly. The point source has no intrinsic frequency dependence, but its dirty-image response contains the spectral phase factors $exp\left(2\pi i\nu {\tau }_j\right)$. Its reconstructed image and sidelobes, therefore, oscillate with frequency solely because the interferometric $\mathrm{PSF}$ changes across the band. The angular quantity ${\mathit{b}}_j·(\mathit{\theta }-{\mathit{\theta }}_s)$ has been converted into the spectral phase $2\pi \nu {\tau }_j$. This is the image-space representation of the second term in Equation (16).

The delay transform provides an equivalent interpretation. For an infinitely wide observing band, the contribution from the jth baseline would be localized at $\tau ={\tau }_j$. Restricting the observation to a finite top-hat band multiplies the frequency spectrum by a rectangular window. Its Fourier transform is a sinc function, so each geometric-delay contribution is broadened into a sinc profile centred at ${\tau }_j$. The first zeros occur at $|\tau -{\tau }_j|=1/\Delta \nu$, while the sidelobes extend beyond the central geometric delay. This delay-space structure is not intrinsic to the source: its centre is set by the chromatic interferometric fringe, and its width and sidelobes are set by the finite observing band.

Equation (17) uses an idealized unweighted sampling function to expose the basic geometry. In a real dirty-map reconstruction, the sampled visibilities are additionally multiplied by imaging weights before the inverse Fourier transform. Natural, uniform, and robust weighting produce different effective sampling functions and, therefore, different PSF main lobes and sidelobes. Frequency-dependent flagging or weighting can introduce additional chromaticity. These choices modify the detailed strength and shape of mode mixing, but they do not remove its fundamental origin: ${\mathit{u}}_j\left(\nu \right)=\nu {\mathit{b}}_j/c$.

To connect the interferometric coordinates to cosmological Fourier modes, consider a narrow observing band centred at ${\nu }_{\mathrm{obs}}$, corresponding to redshift $z={\nu }_{21}/{\nu }_{\mathrm{obs}}-1$, where ${\nu }_{21}$ is the rest-frame frequency of the 21 cm transition. An angular Fourier coordinate $\mathit{u}$ maps to the transverse comoving wavenumber $k_{\perp }$, while the delay variable $\tau$ maps approximately to the line-of-sight wavenumber $k_{\Vert }$:

$$k_{\perp }={\displaystyle \frac{2\pi \left|\mathit{u}\right|}{D_M\left(z\right)}},k_{\Vert }={\displaystyle \frac{2\pi \tau }{Y\left(z\right)}},\mathrm{with}Y\left(z\right)\equiv \left|{\displaystyle \frac{\mathrm{d}\chi }{\mathrm{d}\nu }}\right|={\displaystyle \frac{c{(1+z)}^2}{H\left(z\right){\nu }_{21}}}.$$

(19)

Here, $D_M\left(z\right)$ is the transverse comoving distance, $\chi$ is the line-of-sight comoving distance, $H\left(z\right)$ is the Hubble parameter, and $Y\left(z\right)$ converts a frequency interval into a radial comoving-distance interval. Therefore, increasing baseline length corresponds to increasing $k_{\perp }$, whereas increasing geometric delay corresponds to increasing $k_{\Vert }$.

For emission within an angle ${\theta }_{max}$ of the phase centre, the  geometric delay on a baseline of length $\left|\mathit{b}\right|$ is bounded approximately by $\left|\tau \right|\lesssim \left|\mathit{b}\right|sin{\theta }_{max}/c$. Substituting this bound into Equation (19), gives

$$|k_{\Vert }|\lesssim {\displaystyle \frac{H\left(z\right)D_M\left(z\right)}{c(1+z)}}sin{\theta }_{max}{k}_{\perp }.$$

(20)

For the full visible sky, ${\theta }_{max}=\pi /2$ and the delay bound becomes the horizon limit $\left|\tau \right|\le \left|\mathit{b}\right|/c$. Equation (20) is, therefore, a straight-line boundary in the $(k_{\perp },k_{\Vert })$ plane. Longer baselines probe larger $k_{\perp }$ and simultaneously permit smooth foreground emission to reach larger $k_{\Vert }$, producing the wedge-shaped foreground region.

Figure 4 summarizes how instrumental chromaticity converts angular foreground structure into spectral structure. In panel (a), the intrinsic point-source spectrum is constant, whereas the dirty-image response varies with frequency because the interferometric PSF is chromatic. Panel (b) shows the same effect in delay space. Each baseline contributes near its geometric delay ${\tau }_j$, and the finite top-hat bandwidth broadens this contribution into a sinc response with extended sidelobes. In panel (c), baseline length is mapped to $k_{\perp }$ and delay is mapped to $k_{\Vert }$. Since the maximum geometric delay increases with baseline length, smooth foreground power occupies a wedge-shaped region in the cylindrical power spectrum. The weaker power extending above the ideal boundary is produced by the finite-band sinc sidelobes, while additional instrumental chromaticity can further broaden this leakage in real observations.

Figure 4. Illustration of instrumentally induced mode mixing for an intrinsically flat-spectrum point source. (a) The chromatic PSF produces a frequency-dependent dirty-image response. (b) A finite bandwidth broadens the geometric-delay contributions into sinc responses. (c) Mapping baseline length and delay to $k_{\perp }$ and $k_{\Vert }$ produces the foreground wedge. The cyan and white dashed lines denote the ${30}^{\circ }$ beam-limited and horizon boundaries, respectively. The colour scale shows ${log}_{10}(P/P_{max})$, where P is the simulated foreground power at each $(k_{\perp },k_{\Vert })$ mode and $P_{max}$ is its maximum value over the displayed Fourier plane.

4.2. Chromatic Mode Mixing in the Image Domain

In the image domain, we describe the interferometric response in terms of PSF, which is closely related to the synthesized beam discussed above. The use of “PSF” here emphasizes the response of the imaging operator, including the effect of the adopted visibility weighting. Interferometric visibilities can be converted into image cubes using, for example, natural or uniform weighting, providing substantial data compression and a more intuitive representation for subsequent analysis. The frequency dependence of this imaging response then produces chromatic mode mixing in the reconstructed image cube.

The foreground wedge is produced by the same chromatic sampling effect whether the analysis is performed in visibility space or in the image domain. In an image-based analysis, the visibilities at each frequency are weighted and assigned to a regular $uv$ grid before an inverse angular Fourier transform is used to construct the dirty-image cube $I_{\mathrm{D}}(\mathit{\theta },\nu )$. At a given frequency, this relation can be written schematically as the angular convolution of the primary-beam-weighted sky brightness with the interferometric PSF. Because both the sampled $uv$ coordinates and the imaging weights depend on frequency, the synthesized PSF is chromatic. A spectrally smooth sky can, therefore, produce a dirty-image cube that is not smooth along the frequency direction.

In the following, we use simulated dirty-map and PSF cubes from the SKA Science Data Challenge 3a, generated using natural weighting, to demonstrate the mode-mixing effect in the image domain [47]. Figure 5 shows this effect in individual frequency channels. The upper row presents dirty-map slices, and the lower row shows the corresponding synthesized PSFs. As the PSF pattern changes across the observing band, the apparent source morphology and sidelobe structure in the dirty map also change. The resulting frequency dependence is, therefore, not necessarily intrinsic to the sky.

Figure 5. Dirty-map and synthesized-PSF slices at selected frequencies, illustrating the chromatic image-domain response.

The same behaviour is more clearly exposed by the frequency–position cuts in Figure 6, where a fixed spatial cut is extracted from every channel and arranged along the frequency axis. Channel-dependent structure in the dirty-map cut follows the corresponding variation of the PSF. Fourier transforming $I_{\mathrm{D}}(\mathit{\theta },\nu )$ along frequency, therefore, maps this instrumentally generated spectral structure into non-zero $k_{\Vert }$ modes. When combined with the transverse modes sampled by the interferometer, this produces the wedge in $P_{2\mathrm{D}}(k_{\perp },k_{\Vert })$.

Figure 6. Frequency–position cuts through the dirty-map and PSF cubes, showing non-smooth chromatic structure.

The common origin of the visibility- and image-domain descriptions is most easily understood in three-dimensional $(u,v,\nu )$ space. Measuring a pure line-of-sight mode at fixed transverse scale would require sampling along frequency while keeping $(u,v)$ fixed, corresponding to a trajectory perpendicular to the $uv$ plane. A physical baseline instead satisfies $\mathit{u}\left(\nu \right)=\nu \mathit{b}/c$ and, therefore, follows an oblique trajectory: changing frequency also changes the transverse Fourier mode being sampled. A single baseline consequently mixes line-of-sight spectral variation with angular Fourier structure.

Gridding attempts to form approximately vertical frequency columns at fixed $(u,v)$ by combining measurements from different baselines. However, the  baselines, flagged samples, visibility densities, and effective weights contributing to a given $uv$ cell generally vary with frequency. Natural or uniform weighting changes the detailed PSF, sidelobes, and leakage amplitude, but does not remove the underlying baseline migration. Thus, even a point source with a perfectly flat intrinsic spectrum can acquire spectral structure through the chromatic interferometric response. The wedge is, therefore, an instrument-induced manifestation of angular–spectral mode mixing, schematically $k_{\perp }\to k_{\Vert }$. Primary-beam chromaticity, frequency-dependent flagging, calibration residuals, cable reflections, polarization leakage, and other instrumental effects can further broaden the contaminated region beyond the ideal geometric boundary [48,49,50,51,52].

5. Bayesian Foreground and Signal Inference

The component-separation methods discussed in Section 3 are primarily designed to construct a foreground estimate or a cleaned residual. Their output is often treated as a single reconstructed field, and uncertainties associated with foreground subtraction, signal loss, and instrumental modelling must then be estimated separately. Bayesian methods instead formulate foreground removal as an inference problem in which the cosmological signal, foregrounds, and  instrumental response are latent quantities. Component separation is, therefore, not eliminated; rather, it is embedded within a generative model whose degeneracies and uncertainties can, in principle, be propagated into the reconstructed 21 cm field and its summary statistics.

5.1. A Hierarchical Forward Model

Let $\mathit{d}$ denote the observed data, which may consist of image-domain temperatures or interferometric visibilities. A general measurement equation can be written as

$$\mathit{d}=\mathcal{R}\left(\mathit{\varphi }\right)\left(\mathit{s}+\mathit{f}\right)+\mathit{n},$$

(21)

where $\mathit{s}$ is the cosmological 21 cm field, $\mathit{f}$ contains the foreground components, $\mathit{n}$ is instrumental noise, and  $\mathcal{R}\left(\mathit{\varphi }\right)$ is the measurement operator. The instrumental parameters $\mathit{\varphi }$ may describe the primary beam, complex gains, baseline coordinates, polarization leakage, mutual coupling, flagging, and other properties of the telescope. Conventional component separation frequently approximates $\mathcal{R}$ as known and attempts to estimate $\mathit{f}$ before analysing the residual. A joint Bayesian analysis instead treats at least some of the quantities entering $\mathcal{R}$ as uncertain model parameters.

Collecting the unknowns into $\mathbf{\Theta }=\{\mathit{s},\mathit{f},\mathit{\varphi },{\mathit{\alpha }}_s,{\mathit{\alpha }}_f\}$, where ${\mathit{\alpha }}_s$ and ${\mathit{\alpha }}_f$ denote signal and foreground hyperparameters, Bayes’ theorem gives

$$p(\mathbf{\Theta }\mid \mathit{d})={\displaystyle \frac{p(\mathit{d}\mid \mathit{s},\mathit{f},\mathit{\varphi })p(\mathit{s}\mid {\mathit{\alpha }}_s)p(\mathit{f}\mid {\mathit{\alpha }}_f)p\left(\mathit{\varphi }\right)p\left({\mathit{\alpha }}_s\right)p\left({\mathit{\alpha }}_f\right)}{\mathcal{Z}\left(\mathit{d}\right)}}.$$

(22)

Here $\mathcal{Z}\left(\mathit{d}\right)$ is the Bayesian evidence. If the real and imaginary parts of the visibilities are represented as a single real data vector and the noise is Gaussian with covariance $\mathit{N}$, the likelihood is

$$p(\mathit{d}\mid \mathbf{\Theta })\propto {\left|\mathit{N}\right|}^{-1/2}exp\left[-{\displaystyle \frac{1}{2}}{\mathit{r}}_{\mathbf{\Theta }}^{\mathsf{T}}{\mathit{N}}^{-1}{\mathit{r}}_{\mathbf{\Theta }}\right],{\mathit{r}}_{\mathbf{\Theta }}=\mathit{d}-\mathcal{R}\left(\mathit{\varphi }\right)(\mathit{s}+\mathit{f}).$$

(23)

The desired signal posterior is obtained by marginalizing over the nuisance parameters,

$$p(\mathit{s},{\mathit{\alpha }}_s\mid \mathit{d})=\int p(\mathbf{\Theta }\mid \mathit{d})\mathrm{d}\mathit{f}\mathrm{d}\mathit{\varphi }\mathrm{d}{\mathit{\alpha }}_f.$$

(24)

A reconstructed map may then be represented by the posterior mean ${⟨\mathit{s}⟩}_{p(\mathit{s}\mid \mathit{d})}$ or by the maximum-a-posteriori solution. More importantly, the power spectrum can be evaluated for every posterior draw of $\mathit{s}$. This propagates foreground and instrumental uncertainties into credible intervals on $P_{21}\left(k\right)$ instead of attaching thermal-noise error bars to a single foreground-subtracted map.

Figure 7 summarizes the end-to-end Bayesian framework. The 21 cm signal, foreground emission, and uncertain instrumental parameters are propagated through a common telescope-response model to predict the measured data. The likelihood must be evaluated in the same data domain as the observations. In a visibility-domain analysis, the  forward model predicts interferometric visibilities directly. In an image-domain analysis, the predicted visibilities are additionally gridded, weighted, and Fourier transformed to produce a model dirty-image data cube before comparison with the observed cube. The latter approach must, therefore, retain the frequency-dependent sampling function, imaging weights, synthesized PSF, and correlated image-domain noise within the measurement model.

Figure 7. End-to-end Bayesian inference of the 21 cm signal, foregrounds, and instrumental response. The forward model predicts either interferometric visibilities or a dirty-image data cube, while marginalization propagates foreground and instrumental uncertainties into the reconstructed signal and power spectrum.

Posterior samples yield distributions for both the 21 cm field and its power spectrum rather than one foreground-subtracted estimate. Kern [53] gives a visibility-domain implementation in which model visibilities are compared directly with the observed visibilities.

5.2. From Gaussian-Process Models to Joint Field Inference

Gaussian-process regression provides one of the clearest early examples of this probabilistic viewpoint. At a given angular position or visibility mode, the foreground, 21 cm signal, and noise are assigned covariance matrices. For  fixed kernel hyperparameters, their conditional mean and covariance follow Equation (9), with the foreground, signal, and noise kernels identified with the corresponding components of the hierarchical model. Foreground smoothness is encoded through the correlation length and functional form of ${\mathit{K}}_{\mathrm{fg}}$, while a shorter correlation scale is normally assigned to the 21 cm component. Kernel hyperparameters can be selected by maximizing the marginal likelihood or sampled jointly with the latent components [12,40]. The former is often described as empirical Bayes: it retains the conditional uncertainty of the latent fields but does not fully marginalize over uncertainty in the optimized hyperparameters. In either case, an inappropriate kernel can absorb part of the 21 cm signal or leave foreground residuals, so Bayesian terminology alone does not guarantee protection against signal loss.

An important early field-level framework was HIEMICA, the HI Expectation-Maximization Independent Component Analysis method [54]. HIEMICA models the measured three-dimensional field as a mixture of statistically independent latent components. Its expectation step computes the conditional moments of the foreground and 21 cm fields for the current mixing and covariance parameters, while its maximization step updates the foreground frequency-mixing functions, foreground angular power spectra, and the three-dimensional 21 cm power spectrum by maximizing the expected complete-data likelihood. It is, therefore, a Bayesian semi-blind extension of ICA and an important bridge between blind component separation and hierarchical inference. However, its original demonstrations adopted idealized instrumental assumptions, and the EM optimization produces point estimates of the hyperparameters rather than a fully sampled joint posterior over all foreground, signal, and instrumental degrees of freedom.

Subsequent visibility-domain analyses moved closer to joint inference through the full measurement equation. BayesEoR jointly estimates foreground coefficients, 21 cm fluctuations, and power-spectrum parameters while forward-modelling their contribution to interferometric visibilities [55]. Gaussian constrained realizations combined with Gibbs sampling provide another scalable route: conditional draws of the 21 cm field, foreground modes, and covariance or power-spectrum parameters are generated in turn, allowing uncertain foreground modes and missing frequency channels to be marginalized rather than deterministically interpolated [56]. Related Gibbs frameworks have been extended to low-resolution full-sky foreground-map reconstruction from visibilities [57].

These studies also demonstrate that the angular extent of the forward model matters. A foreground model confined to the primary field of view can fail to represent bright emission entering through chromatic sidelobes. Tests with BayesEoR found that such unmodelled all-sky emission can generate strong spectral contamination and, in severe cases, numerical instability; an all-sky foreground model with a more restricted 21 cm model was introduced to reduce the computational cost [58]. Further injection tests show that Bayesian power-spectrum recovery can remain robust over a substantial fraction of high-delay modes even when the foreground catalogue is perturbed, but the most foreground-dominated low-delay modes remain sensitive to foreground parametrization [59]. These results emphasize that a Bayesian posterior is conditional on the assumed model family: an incomplete forward model can produce a biased, and potentially overconfident, posterior.

5.3. Differentiable End-to-End Inference

Differentiable programming extends Bayesian inference through the telescope response itself. Automatic differentiation provides gradients of the posterior with respect to sky and instrumental parameters, enabling efficient gradient-based optimization and sampling without relying on finite differences [60].

Ref. [53] implements this idea in the BayesLIM framework. In a mock HERA analysis, the 21 cm field, the full-sky foreground emission, and the frequency-dependent primary beam are inferred jointly in a model containing approximately $8.8\times {10}^4$ complex-valued parameters. A staged L-BFGS optimization is followed by Hessian-preconditioned Hamiltonian Monte Carlo, and the resulting posterior samples are propagated into 21 cm maps and power spectra.

The principal result is a marginalized uncertainty that includes foreground–signal and instrument–signal degeneracies. In the simulated analysis, the uncertainty increases substantially at low k, where the foreground, beam, and 21 cm components become difficult to distinguish. Thermal-noise errors alone would, therefore, underestimate the uncertainty in these modes. The current calculation is nevertheless a proof of concept: it models an antenna-independent primary beam but keeps several other effects, including complex gains, polarization leakage, and mutual coupling, fixed or absent.

5.4. Benefits and Limitations of Field-Level Bayesian Inference

Modes that cannot be cleanly separated are represented by broader or correlated posterior distributions rather than being assigned entirely to one component. This is the main statistical difference between joint inference and a single foreground-subtracted residual.

This Bayesian framework incorporates prior information explicitly. External sky maps, beam measurements, calibration data, and theoretical signal models can regularize poorly constrained modes, while the dependence on these assumptions can be tested by varying the priors. Because the telescope response enters the forward model, chromaticity, irregular sampling, and flagged channels can be included directly in the likelihood instead of being treated only through post-processing corrections.

Bayesian evidence further provides a formal way to compare foreground, instrumental, and signal models [61]. In practice, however, evidence calculation is often much more expensive than parameter estimation for high-dimensional visibility models. Bayesian analyses should, therefore, not be expected to produce smaller uncertainties; their benefit is that the reported uncertainties more completely represent the information and degeneracies present in the data.

Field-level Bayesian inference is computationally demanding because the forward model and its gradients must be evaluated many times. As a representative example, the BayesLIM proof-of-concept sampling required approximately 64 GPU hours, while its automatic-differentiation graph occupied more than 200 GB of aggregate GPU memory [53]. More realistic models containing antenna-dependent beams, calibration parameters, polarization, mutual coupling, and higher-resolution skies will further increase this cost.

A second difficulty is the geometry of the posterior. The 21 cm signal, foregrounds, and chromatic instrumental response can produce similar changes in the measured visibilities, leading to strongly correlated parameters, poorly conditioned curvature, or nearly singular Hessian matrices. Kern [53], for example, removed a subset of poorly constrained 21 cm modes that would otherwise make the Hessian singular without strong regularization, and  found slower posterior mixing for the beam parameters than for the foreground and 21 cm components.

Low signal-to-noise ratio should not, by itself, be identified with numerical non-convergence. When the signal is weak, the likelihood provides little information in the corresponding directions, so the posterior becomes broad and increasingly determined by the prior. Such a posterior may still converge numerically. Slow mixing or an apparently “stuck” sampler is more likely when low signal-to-noise modes are also strongly degenerate with foreground or instrumental parameters, or when the parametrization produces directions with very different curvature. Convergence should, therefore, be assessed using multiple chains, autocorrelation lengths, effective sample sizes, acceptance rates, and sensitivity to initialization and prior width.

Finally, Bayesian marginalization remains conditional on the assumed model. Overly restrictive priors can suppress real 21 cm fluctuations, while an incomplete foreground or instrumental model can produce biased and overconfident posteriors [58,59]. Signal-injection tests, posterior-predictive checks, prior-sensitivity analyses, and end-to-end simulations are consequently essential. The Bayesian framework does not automatically separate fundamentally degenerate components; its central advantage is that, when the forward model is adequate, those degeneracies are made explicit and propagated into the final cosmological uncertainty.

6. Deep Learning for Foreground Mitigation and Signal Reconstruction

Deep-learning methods replace a prescribed linear foreground model with a nonlinear mapping learned from simulations. Given an observed data cube $\mathit{y}$ and an optional preprocessing operator $\mathcal{P}$, the recovered 21 cm field can be written schematically as

$$\widehat{\mathit{s}}={\mathcal{N}}_{\psi }\left(\mathcal{P}\mathit{y}\right),$$

(25)

where $\mathit{y}$ is the observed or preprocessed data cube and ${\mathcal{N}}_{\psi }$ is a neural network with trainable parameters $\psi$. Training samples contain simulated contaminated cubes and their corresponding true 21 cm fields. The network is commonly optimized using a voxel-level reconstruction loss, sometimes supplemented by losses based on the power spectrum, cross-correlation coefficient, or other summary statistics. Unlike line-of-sight methods, three-dimensional networks can use angular morphology and correlations between frequency channels simultaneously.

6.1. Convolutional Autoencoders and U-Nets

Early applications used convolutional denoising autoencoders, in which an encoder compresses the contaminated image into a latent representation and a decoder reconstructs the cosmological signal. Using simulated SKA observations, Li et al. [62] showed that such a network could model foreground structure complicated by a frequency-dependent beam more flexibly than polynomial or wavelet-based fitting.

The deep21 framework extended this approach using a three-dimensional U-Net [63]. Its encoder extracts spatial–spectral features on progressively larger scales, while the decoder reconstructs the full-resolution 21 cm cube. Skip connections transfer fine-scale information between corresponding encoder and decoder layers, reducing the loss of localized structures during compression. This architecture can exploit information that is not captured by a small number of linear foreground modes and can recover clustering statistics more accurately than standard PCA in idealized simulations. Ensembles of independently trained networks can provide an empirical estimate of prediction variance, although they do not by themselves account for uncertainty caused by an incorrect training model.

Figure 8 illustrates the basic U-Net reconstruction process. Convolution and downsampling compress the foreground-dominated input into a multi-scale latent representation. The  decoder then combines these features with spatial information transferred through the skip connections to reconstruct the 21 cm brightness-temperature field.

Figure 8. Schematic U-Net reconstruction of the 21 cm field from an observed data cube using multi-scale spatial–spectral features and skip connections.

6.2. Frequency Differencing and PCA-Assisted Networks

Applying a neural network directly to the raw data is difficult because the foreground can exceed the 21 cm fluctuations by several orders of magnitude. An initial linear transformation can reduce this dynamic range before neural reconstruction. Two commonly used inputs are frequency-differenced data and PCA residuals:

$$\begin{array}{cccc}\hfill {\mathit{y}}_{\mathrm{FD}}\left({\nu }_i\right)& =\mathit{y}\left({\nu }_{i+1}\right)-\mathit{y}\left({\nu }_i\right),\hfill & \hfill {\mathit{y}}_{\mathrm{PCA}}& =\mathit{y}-{\mathit{U}}_m\left({\mathit{U}}_m^{\mathsf{T}}\mathit{y}\right),\hfill \end{array}$$

(26)

where the columns of ${\mathit{U}}_m$ contain the first m foreground-dominated PCA modes. Frequency differencing suppresses components that vary slowly between adjacent channels without requiring the number of removed foreground modes to be specified [64]. PCA usually removes more foreground power, but it can also project out cosmological modes that overlap with the foreground subspace.

The two transformations retain partially complementary information. Wang et al. [65] used the FD and PCA residual cubes as separate input channels of a three-dimensional U-Net. The FD channel helps preserve information suppressed by PCA, whereas the PCA channel provides a cleaner initial foreground subtraction. The network combines these inputs and learns a nonlinear correction for their residual biases. This illustrates that deep learning does not need to replace traditional filters; it can instead combine their outputs to improve signal reconstruction.

6.3. Reconstruction of Foreground-Wedge Modes

Deep learning can also be applied after foreground avoidance. Removing the contaminated wedge in $(k_{\perp },k_{\Vert })$ space produces a cleaner but incomplete 21 cm field. Because the EoR signal is non-Gaussian, the retained and removed Fourier modes are not statistically independent. Neural networks can learn these higher-order correlations from simulations and infer morphological information associated with the missing modes.

Gagnon-Hartman et al. [66] demonstrated that a U-Net could recover the positions, shapes, and approximate sizes of large ionized regions from wedge-filtered images. This approach was subsequently extended to foreground-filtered light cones and high-redshift galaxy mapping [67]. These outputs should be interpreted as conditional reconstructions rather than direct measurements of the removed modes. Their accuracy necessarily depends on the range of astrophysical models represented in the training simulations.

6.4. Segmentation and Morphological Reconstruction

For EoR tomography, the scientific objective may be to identify neutral and ionized regions rather than to recover every brightness-temperature value exactly. Segmentation networks classify pixels or voxels according to their likely ionization state and can, therefore, remain useful when the reconstructed signal amplitude is imperfect.

SegU-Net v2 has been tested on images processed using PCA, GPR, polynomial fitting, and wedge removal. These tests showed that ionized-region morphology can remain recoverable after several different foreground treatments [68]. The later SERENEt framework directly reconstructs the 21 cm brightness-temperature field and can incorporate prior information about the likely locations of ionized regions [69]. Such priors may be obtained from a segmentation network or from overlapping high-redshift galaxy surveys. External tracers can, therefore, help reduce ambiguities between foreground residuals and genuine ionized structures.

6.5. Generative and Field-Level Reconstruction

A deterministic U-Net returns a single reconstructed field, often close to the conditional mean of the possible 21 cm realizations. This can smooth small-scale structures and does not represent the range of fields consistent with the observed data. Generative methods instead approximate a conditional distribution and produce multiple plausible reconstructions.

Sabti et al. [70] used a stochastic-interpolant generative model to reconstruct three-dimensional 21 cm light cones after wedge filtering. Repeated sampling produces an ensemble of possible fields conditioned on the retained Fourier modes. This ensemble can be propagated to power spectra, cross-correlations, bubble statistics, and other derived quantities.

A complementary approach is field-level inference. Chen et al. [71] developed a differentiable physical forward model that maps initial matter fluctuations to foreground-filtered 21 cm maps, allowing the initial conditions and bias parameters to be inferred jointly. The same work also investigated diffusion-based map reconstruction. These methods make the ambiguity of the missing modes explicit: when the data do not uniquely determine the field, the appropriate result is a posterior or ensemble of physically plausible reconstructions rather than one apparently exact map.

6.6. Advantages and Limitations

Compared with PCA, ICA, GMCA, GPR, and DAYENU, deep-learning methods can use nonlinear, non-Gaussian, and multi-scale spatial–spectral information. They can combine several preprocessed data products, incorporate external tracers, and partially reconstruct information removed by foreground avoidance. Once trained, they also provide rapid field-level predictions.

The main limitation is their dependence on simulations. A network can learn correlations specific to the assumed foreground, beam, calibration, noise, or astrophysical model and may become biased when applied outside its training distribution. A small reconstruction loss on simulated validation data does not demonstrate that the same model will be reliable on observations. End-to-end simulations must, therefore, span plausible foreground and instrumental uncertainties. Signal-injection tests, scale-dependent transfer functions, comparisons with conventional foreground-cleaning pipelines, and tests on unseen simulation models are also required. For generative methods, posterior coverage and uncertainty calibration must be verified explicitly.

Deep learning is, therefore, most useful as a complement to physical foreground and instrumental modeling rather than as a replacement for them. Recent work combines learned reconstruction with sky-model subtraction, forward models that include the telescope response, foreground filtering, and simulation-derived transfer functions. The SKA Science Data Challenge 3a illustrates both the potential of such hybrid pipelines and the scale-dependent residuals that must be controlled before cosmological interpretation [47].

7. Lessons from the SKA Science Data Challenge 3a

The SKA Science Data Challenge 3a (SDC3a) marked an important transition from foreground-removal tests on idealized sky cubes to an end-to-end interferometric analysis. The data were simulated in visibility space and propagated through a realistic SKA-Low response. Participants, therefore, had to recover the EoR power spectrum from visibilities or dirty-image products affected by foregrounds, PSF chromaticity, calibration residuals, ionospheric perturbations, and thermal noise [47]. The main value of SDC3a is not the ranking of individual pipelines, but the opportunity to determine which methodological ideas remain effective once the assumptions of perfect spectral smoothness and ideal imaging are relaxed.

7.1. An End-to-End Benchmark

The simulation used the 512-station SKA-Low array over 106–$196\mathrm{MHz}$, divided into 900 frequency channels, with thermal noise scaled to approximately $1000\mathrm{h}$ of observation. The sky contained diffuse Galactic emission, bright and faint extragalactic sources, strong sources outside the main field of view, and an evolving EoR signal generated with 21cmFAST [6]. More than ${10}^7$ in-field sources were included, while bright off-axis sources were propagated through the far sidelobes and only partially removed to represent imperfect source de-mixing.

The measurement model included frequency-dependent baseline sampling, primary-beam and PSF chromaticity, residual direction-independent gain errors, ionospheric phase perturbations, and thermal noise. Participants were given visibility data, naturally weighted dirty-image and PSF cubes, uniformly weighted and tapered image products, and a frequency-dependent primary beam. The requested products were six cylindrical power spectra over non-overlapping $15\mathrm{MHz}$ intervals, together with their uncertainties. SDC3a, therefore, tested the full chain from source subtraction and imaging to noise-bias correction, beam normalization, Fourier-space binning, and uncertainty estimation [47].

7.2. From Individual Algorithms to Multi-Stage Pipelines

A major result of SDC3a is that realistic foreground mitigation cannot usually be performed by a single component-separation algorithm. Most teams first reduced the instrumental complexity of the data and only then applied PCA, ICA, polynomial fitting, GPR, tensor decomposition, delay filtering, or machine learning. In visibility-domain pipelines, bright off-axis and in-field compact sources were modelled and subtracted before imaging. Some pipelines also removed diffuse Galactic emission. This lowered the dynamic range and reduced the chromatic sidelobe structure presented to the final statistical cleaner.

The three DOTSS-21cm pipelines provide a clear example. Their common preprocessing iteratively subtracted bright off-axis sources and progressively fainter compact sources from the visibilities; the most complete version also removed multiscale diffuse Galactic emission. Two variants then used maximum-likelihood Gaussian process regression (ML-GPR), with separate covariance terms for intrinsic foregrounds, instrumental mode mixing, thermal noise, and the EoR signal. The third instead excluded foreground-dominated Fourier modes. GPR performed better after the dominant foreground structure had been removed physically, whereas avoidance required fewer covariance assumptions but discarded substantially more cosmological information [47].

Other pipelines applied PCA, ICA, polynomial fitting, or GPR directly to dirty or partially deconvolved image cubes. Their performance depended strongly on the preceding image formation. Even if the intrinsic foreground spectrum is smooth, convolution with a frequency-dependent PSF can make the dirty-image cube spectrally complex and increase its effective rank. A low-rank or smooth-kernel model applied without accounting for the PSF can, therefore, leave foreground residuals or remove part of the EoR signal.

Several teams worked partly or entirely in delay space, identifying and excluding modes dominated by foreground emission. These methods were less dependent on a detailed foreground model, but remained vulnerable to supra-horizon leakage and sacrificed cosmological information. Bayesian forward-modelling approaches instead described the EoR signal, residual foregrounds, and noise jointly in delay or visibility space and propagated them into a power-spectrum posterior.

7.3. Instrumental Preprocessing and Learned Methods

SDC3a also tested several recently developed algorithms on substantially more realistic data than those used in most methodological demonstrations. The HIMALAYA pipeline reconvolved the dirty images to make the effective PSF more uniform across frequency before applying PCA. Rather than requiring PCA to model the full chromatic instrumental response, the preprocessing made the data more closely satisfy the low-rank foreground assumption. The Shuimu–Tianlai pipeline used oriented singular value decomposition (O-SVD), treating the frequency–angular cube as a third-order tensor and removing dominant foreground modes without first flattening the cube into a matrix. These methods illustrate that an effective innovation can consist of transforming the data into a representation in which a relatively simple separation model becomes valid again.

Deep-learning pipelines used U-shaped convolutional networks, semisupervised deconvolution, or transformer encoders. ERWA combined a U-Net with polynomial foreground fitting; SKACH added a three-dimensional network after visibility source subtraction and smooth residual cleaning; and Hausos used a transformer–convolutional architecture. In REACTOR, the network prediction was reconvolved with the PSF during training so that the reconstructed sky remained consistent with the dirty image [47].

These experiments showed that neural networks can represent nonlinear and spatially varying instrumental effects that are difficult to capture with a fixed polynomial or stationary covariance kernel. However, their accuracy depended strongly on whether the training simulations reproduced the PSF, noise, gain errors, ionosphere, source population, and frequency coverage of the challenge data. Patch-based training could lose large-scale information, while inappropriate output constraints could bias fields containing both positive and negative brightness-temperature fluctuations. The SDC3a results, therefore, show that learned reconstructions must include the measured telescope response and be tested on simulations not used for training; a direct mapping from dirty images to clean EoR fields is not sufficient.

7.4. Main Lessons

All submitted pipelines suppressed the foreground power by at least approximately one order of magnitude, and several independent methods obtained median residuals below the injected EoR power. Nevertheless, no method was unbiased over every Fourier scale and frequency interval. Some pipelines retained positive foreground residuals, whereas others underestimated the power through EoR signal loss. Because the biases of different methods often had different signs, agreement between independent pipelines provides a useful diagnostic and may reduce the dependence on any single foreground model [47].

The central physical lesson is that foreground removal cannot be separated from calibration and imaging. Much of the difficult spectral structure came from the interaction of bright sources with a chromatic PSF rather than from intrinsically non-smooth emission. Successful pipelines removed those sources in visibility space or regularized, deconvolved, or reconvolved the instrumental response before applying PCA, polynomial fitting, ICA, or GPR.

A second lesson is that a visually clean residual image is not sufficient. Foreground cleaning can attenuate the EoR signal in a scale- and frequency-dependent manner. Transfer functions derived from signal injections or end-to-end simulations are, therefore, required. Cross-power spectra constructed from independent time splits can avoid the mean thermal noise bias, but they do not by themselves correct foreground residuals, calibration errors, or signal loss.

Uncertainty estimation proved more difficult than foreground suppression. Most pipelines underestimated the actual power-spectrum errors, sometimes substantially; even the most closely calibrated submitted uncertainties accounted for only about $60\%$ of the true errors on average. Thermal-noise errors alone are, therefore, insufficient for a detection claim. Real analyses will require ensembles of systematic realizations, signal injections, transfer-function uncertainties, and comparisons between independent pipelines.

SDC3a also showed that the simulations require independent validation. A frequency-scaling error affecting several components was discovered after the challenge, and corrected products were released. The error preserved the foreground smoothness relevant to the exercise, but it reinforces the need for versioned and reproducible simulations; the challenge also omitted some important effects, notably polarized leakage into Stokes I. Overall, EoR recovery should be treated as an end-to-end inverse problem linking source modelling, calibration and imaging, component separation, transfer-function calibration, power-spectrum estimation, and uncertainty propagation.

8. Validation and Outlook

End-to-end recovery must be tested against the injected field as well as its summary statistics. This section separates phase coherence from amplitude response and uses both to state practical criteria for a faithful reconstruction.

8.1. Cross-Correlation and Field-Level Validation

Power-spectrum agreement alone is insufficient to demonstrate that the EoR signal has been correctly reconstructed at the field level. The auto-power spectrum measures the variance of the recovered Fourier modes but does not retain their phases. A residual foreground or instrumental artefact may, therefore, have approximately the same power as the injected EoR signal while being spatially unrelated to it. For example, a reconstruction dominated by a contaminant q could satisfy $P_{qq}\left(k\right)\simeq P_{ss}\left(k\right)$ but have $P_{qs}\left(k\right)\simeq 0$. Its recovered auto-power spectrum would appear correct, even though the underlying EoR realization had not been recovered.

For simulations in which both the injected field s and reconstructed field $\widehat{s}$ are available, field-level fidelity can be quantified using the map-space correlation coefficient and the scale-dependent Fourier-space coherence:

$$\begin{array}{cccc}\hfill r_{\mathrm{map}}& ={\displaystyle \frac{⟨\left(\widehat{s}-⟨\widehat{s}⟩\right)\left(s-⟨s⟩\right)⟩}{{\sigma }_{\widehat{s}}{\sigma }_s}},P_{\widehat{s}s}\left(k\right)\hfill & \hfill =Re{⟨\tilde{\widehat{s}}\left(\mathbf{k}\right){\tilde{s}}^{,*}\left(\mathbf{k}\right)⟩}_{\left|\mathbf{k}\right|\in k},r\left(k\right)& ={\displaystyle \frac{P_{\widehat{s}s}\left(k\right)}{\sqrt{P_{\widehat{s}\widehat{s}}\left(k\right)P_{ss}\left(k\right)}}}.\hfill \end{array}$$

(27)

where $\mathit{k}$ is the Fourier wavevector and $k\equiv \left|\mathit{k}\right|$ denotes its magnitude. The power spectra are averaged over Fourier modes within a bin centred on k. Here, the averages in $r_{\mathrm{map}}$ are taken over the reconstructed volume, ${\sigma }_{\widehat{s}}$ and ${\sigma }_s$ are the corresponding standard deviations, and the Fourier-space average is taken over the modes within a given k bin. The same definition can be applied to cylindrical bins by replacing k with $(k_{\perp },k_{\Vert })$. The quantity $r\left(k\right)$ lies between $-1$ and 1: a value close to unity indicates that the reconstructed and injected fields are coherent and preserve the same Fourier phases and morphology on that scale, whereas $r\left(k\right)\simeq 0$ indicates little field-level correspondence.

A high correlation coefficient is necessary for phase-level reconstruction, but it does not by itself guarantee the correct signal amplitude. For example, a rescaled reconstruction $\widehat{s}=as$ has $r\left(k\right)=1$ for any positive value of a. The amplitude response can be quantified by the cross-transfer function

$$T\left(k\right)={\displaystyle \frac{P_{\widehat{s}s}\left(k\right)}{P_{ss}\left(k\right)}}.$$

(28)

A faithful reconstruction should, therefore, satisfy $r\left(k\right)\simeq 1$, $T\left(k\right)\simeq 1$, and $P_{\widehat{s}\widehat{s}}\left(k\right)\simeq P_{ss}\left(k\right)$ over the scientifically relevant modes (although, strictly speaking, these conditions are not independent, and satisfying any two is sufficient to imply the third in the ideal limit). These complementary conditions test, respectively, the phase coherence, multiplicative response, and total recovered power.

Including these diagnostics in future simulation challenges would distinguish genuine EoR reconstruction from an accidental agreement in auto-power caused by residual foregrounds, noise, or instrumental systematics. The test is directly applicable to methods that return a reconstructed EoR cube. For pipelines that estimate only the power spectrum, an analogous response must be measured through injected-signal cross-spectra and end-to-end transfer-function tests. In real observations the true EoR realization is unavailable, so signal injections provide the primary calibration.

8.2. Outlook

Field-level EoR reconstruction remains a challenging and highly underconstrained inverse problem. Unlike compact-binary gravitational-wave searches, which can draw on accurate waveform templates, the realization and morphology of the EoR signal are unknown and must be inferred in the presence of much brighter foregrounds and an imperfectly known instrumental response. Further methodological advances may, therefore, be needed to improve the reliability of 21 cm signal recovery. Promising directions include physical forward modelling, foreground-mitigation methods that incorporate the telescope response, machine learning, and Bayesian uncertainty propagation, supported by realistic simulations and signal-injection tests. Recovered signals should be evaluated using not only auto-power spectra but also transfer functions, cross-correlations, and calibrated uncertainties. Although substantial challenges remain, continuing advances in observations, simulations, and statistical inference provide a promising path from statistical detection toward tomographic imaging of Cosmic Dawn and reionization.