Unsupervised Domain Adaptation for Constraining Star Formation Histories

Abstract

In astronomy, understanding the evolutionary trajectories of galaxies necessitates a robust analysis of their star formation histories (SFHs), a task complicated by our inability to observe these vast celestial entities throughout their billion-year lifespans. This study pioneers the application of the Kullback–Leibler Importance Estimation Procedure (KLIEP), an unsupervised domain adaptation technique, to address this challenge. By adeptly applying KLIEP, we harness the power of machine learning to innovatively predict SFHs, utilizing simulated galaxy models to forge a novel linkage between simulation and observation. This methodology signifies a substantial advancement beyond the traditional Bayesian approaches to Spectral Energy Distribution (SED) analysis, which are often undermined by the absence of empirical SFH benchmarks. Our empirical investigations reveal that KLIEP markedly enhances the precision and reliability of SFH inference, offering a significant leap forward compared to existing methodologies. The results underscore the potential of KLIEP in refining our comprehension of galactic evolution, paving the way for its application in analyzing actual astronomical observations. Accompanying this paper, we provide access to the supporting code and dataset on GitHub, encouraging further exploration and validation of the efficacy of the KLIEP in the field.

1. Introduction

In recent times, many transfer problems have arisen, and various methods exist to solve them. We can broadly classify these into three categories. First is supervised domain adaptation, where only a few labeled target data are available [1,2,3,4,5,6]. Second is semi-supervised domain adaptation, where, in addition to these labeled data, a large amount of unlabeled data are available [7,8,9,10,11]. Finally, we have unsupervised domain adaptation, where only unlabeled data are available in the target [12,13,14,15,16]. Researchers have shown that adding target labels can increase the performance of the model [3]. This setting is also the most encountered in practice [17], as it is often possible to label at least a few target samples.

However, there are cases in which obtaining supervised data is not possible. Such is the case of the present astrophysics problem of SFH prediction. Here, we aim to learn the history of our universe through the evolution of its galaxies’ masses throughout their individual histories. We try to obtain this history from the radiation that reaches us on Earth—the Spectral Energy Distribution (SED), a function of the brightness of a galaxy with the wavelength of observation.

As galaxies evolve over billions of years, capturing the complete Spectral Energy Distribution (SED) of any single galaxy over time remains a formidable challenge. Astrophysicists have developed models based on physically motivated mechanisms to construct artificial star formation histories (SFHs) and their corresponding radiation outputs. These models, aiming to establish a reliable correspondence between observed radiation and underlying SFHs, predominantly employ classic Bayesian approaches such as ProSpect [18], MagPhys [19,20], CIGALE [21,22], BAGPIPES [23,24], and Prospector [25,26].

While these Bayesian models are instrumental in inferring SFHs from observed SEDs, they are not without limitations. Their performance can be constrained by the simplifications and assumptions inherent in their design, such as the parametrization of dust models, stellar population synthesis, and the treatment of nebular emission. These simplifications, while necessary for computational feasibility, can lead to discrepancies between predicted and actual SFHs, especially when the models are applied to data beyond their calibration range. Moreover, the evaluation of these models often relies on comparisons with synthetic data or specific observational datasets, which may not fully represent the diversity and complexity of galaxy populations in the universe.

Deep neural networks (DNNs) have emerged as powerful tools in various astronomical domains, demonstrating significant success in addressing complex and large-scale data analysis challenges. In the field of exoplanet discovery, DNNs have been employed to identify exoplanetary signals from vast amounts of stellar light curve data [27,28,29]. Similarly, in the realm of cosmology, DNNs have facilitated the analysis of cosmic microwave background data to extract insights about the early universe [30,31,32]. Moreover, in the study of gravitational waves, these networks have accelerated the detection and characterization of gravitational wave signals, contributing to our understanding of astrophysical events [33,34,35]. The application of DNNs extends to the classification and analysis of galaxies, where they help in morphological classifications [36,37] and redshift estimations [38,39]. Specialized neural networks have been used for improving observing schedules at astronomical observatories [40,41], inferring redshifts from a collection of observed temporal and spectral features of gamma ray bursts [42,43], and extracting probabilistic stellar properties from their observed spectra [44,45], showcasing their versatility and expanding their utility in various branches of astronomy.

The challenge of “domain shift”—the discrepancy between the training domain of models and the target observational domain—further complicates the application of these models to real-world data. This is evident even with DNN-based approaches, like mirkwood [46] and beyond-mirkwood [47], which, while promising, also struggle with domain shift when applied to real galaxies. To address these challenges, this work explores unsupervised domain adaptation, aiming to refine the extraction of SFHs from individual galaxies without the constraints of physical model simplifications and conventional SED-fitting techniques’ parametrizations.

We leverage cosmological, hydrodynamic simulations such as eagle [48], illustristng [49,50] and simba [51], which simulate the universe’s evolution from shortly after the Big Bang to the present day. These simulations offer a framework to study galaxies as proxies for real ones, enabling direct comparisons with observational data and helping to refine our models’ predictive capabilities. Through unsupervised domain adaptation, we seek to enhance the generalization of SFH prediction models, ensuring they remain robust across varying galactic environments, thereby enriching our understanding of the universe’s evolution.

2. Data

Our training and test datasets consist of SEDs from three state-of-the-art cosmological galaxy formation simulations, where the true physical properties are known—including their true star formation histories. Simba [51], Eagle [48,52,53], and IllustrisTNG [54], with 1688, 4697, and 9633 samples, respectively, together constitute a diverse sample of galaxies with realistic growth histories. Specifically, we select galaxies at a redshift of 0—this corresponds to simulated galaxies at the “present” epoch. Each SED is comprised of 20 measurements of the “brightness” of a galaxy at different wavelength, in the form of flux density (with units of Jansky) and are the co-variates or features that we train/learn on. The outputs/labels are star SFH time series vectors with 29 scalar elements, in units of solar mass per year (M_⊙ yr⁻¹).

The SFH for a galaxy informs us about the net stellar mass generated as a function of time—sum of masses of all stars born, less the sum of stellar mass lost in stellar winds as stars age and eventually die. In other words, SFHs are plots of the star formation rates (SFRs) of galaxies against the lookback time, which for galaxies at $z=0$ extends from 0 to the very age of our universe (∼13.8 Gyrs). By stacking (adding) together SFHs of several thousand galaxies from a simulation, and dividing by the volume of the simulated box (i.e., the size of the simulated universe), we can derive the cosmic star formation rate density (CSFRD), which is a well-studied global property of our universe. Astrophysicists aim to tune various physical properties in any given simulation until the CSFRD plot derived from it matches closely to the most widely accepted one derived from observations [55].1

In this study, our analysis is confined to galaxies at a redshift of $z=0$, which represents the current state of the universe. This temporal constraint poses a significant limitation when it comes to deriving the Cosmic Star Formation Rate Density (CSFRD), which requires a comprehensive temporal perspective, encompassing a wide range of redshifts to accurately reflect the universe’s star formation history over billions of years. The CSFRD integrates the star formation rates across all galaxies over time and space, offering a global view of star formation activity throughout the history of the universe. However, with our dataset restricted to $z=0$, we lack the longitudinal depth needed to construct this global metric, as we only observe the galaxies’ most recent star formation activities.

To navigate this limitation, we pivot our analysis to leverage $\Sigma$SFHs, which are the summations of individual star formation histories from a collection of galaxies at a single time point. While $\Sigma$SFHs do not provide the extensive temporal insight that CSFRD offers, they still enable us to aggregate star formation activities across a wide array of galaxies at the present epoch. This approach allows us to capture a snapshot of the cumulative star formation activity within the observable universe at $z=0$, providing valuable insights into the current state of galaxy evolution and the prevailing star formation processes. By focusing on $\Sigma$SFHs, we adapt our methodology to the available data, ensuring that our analysis remains robust and informative despite the inherent constraints of our dataset.

We sequentially train on any two of these simulated datasets, and predict on the third, thus giving us three sets of source- and target domain data and results.

3. Methodology

We consider the problem of the prediction of SFH, where the learner has access to a dataset $X\in {\mathbb{R}}^{n\times p}$ encoding the radiations of n galaxies with p as the number of filters/wavelength-bins, and a dataset $Y\in {\mathbb{R}}^{n\times T}$ giving the corresponding SFH for each galaxy with T as the time length of the history (see Figure 1). Each SED (row of X) is comprised of $p=20$ measurements of the “brightness” of a galaxy at different wavelengths, in the form of flux density (with units of Jansky) (see Figure 2). $T=29$, and the sampling is slightly different for each of the three simulations2.

Below are our pre-processing steps:

• First, we create three sets of experiments to ensure a comprehensive evaluation across all simulations. In each set, we utilize galaxies from two of the three simulations—simba, illustristng, and eagle—as our training and validation sets, and galaxies from the remaining simulation as our test set. This approach is designed to emulate real-world scenarios where the ground truth is unknown, and we must rely on models trained on different but related data. The 9:1 split between training and validation sets was chosen as a conventional starting point in machine learning practice, providing a substantial amount of data for training while reserving a reasonable subset for validation. The choice of this split ratio is somewhat arbitrary and commonly used in the field, offering a balance between learning from a large training set and having enough data to validate the model’s performance effectively. While other split ratios or more complex cross-validation schemes might influence the model’s performance, the impact of such variations is beyond the scope of this initial study and is earmarked for future exploration. This study aims to establish a baseline understanding of how well models trained on one set of simulated galaxies generalize to another, using a straightforward and widely accepted split ratio for initial experiments.
• Second, we normalize each SFH time series (each row of Y) by its sum and store the resultant normalized SFH (SFH_norm) and the sum (SFH_sum) separately. This normalization is crucial given the vast dynamic range of star formation histories observed across different galaxies. See Appendix D): SFH curves have a large variety of scales, some increasing to more than 100, whereas others never increase over $0.1$. Without normalization, galaxies with intrinsically high star formation rates could dominate the learning process, overshadowing subtler trends in galaxies with lower rates. By scaling each SFH time series, we facilitate the model’s ability to learn the underlying patterns in the SFHs, independent of their absolute scale. The learning of the SFHs is now decoupled in the learning of SFH_norm and SFH_sum.
  However, this approach also introduces considerations that must be acknowledged. First, the normalization step assumes that the shape of the SFH curve is more informative than its absolute scale, which may not always hold true, especially if the scale itself carries significant astrophysical information. Second, this method may amplify the noise in SFHs with very low sums, potentially impacting the model’s ability to learn meaningful patterns from these histories. Additionally, decoupling the learning of SFH_norm and SFH_sum assumes that these two components are independent, which may not fully capture the complexities of galaxy evolution, where the total star formation and its distribution over time could be interrelated. While this normalization facilitates the learning process by standardizing the range of input features, it is essential to consider these aspects when interpreting the model’s outputs and in future investigations to refine the approach.
• Third, we further ease the learning of the SFH_norm by reducing the curves to their first 3 Kernel-PCA (Principal Component Analysis) components [56,57]. This choice is predicated on a series of comparative analyses where Kernel-PCA demonstrated superior performance in preserving the essential characteristics of the SFH time series when compared to linear PCA [58,59] and discrete wavelet transform (DWT) [60,61]. Our evaluation focuses on the capacity of each method to reconstruct the original time series while maintaining its significant features. The effectiveness of this reconstruction is quantitatively assessed using the DILATE loss metric (Distortion Loss including shApe and TimE) [62], which incorporates both dynamic time warping (DTW) and the temporal distortion index (TDI) to evaluate the similarity between the original and reconstructed time series.
  During our experimentation, Kernel-PCA consistently outperforms its counterparts by achieving lower DILATE loss scores, suggesting that it retains more of the original series’ structural and temporal integrity. Specifically, in our tests, Kernel-PCA exhibits a more robust ability to capture non-linear patterns within the SFH data, a key aspect given the complex nature of these time series. The selection of the number of components (3) is based on a variance-explained criterion, where we aim to retain at least $80\%$ of the variance within the validation set, ensuring a balance between dimensionality reduction and information retention. This process and its outcomes are further detailed in Appendix A and visually represented in Figure A1, where we compare the reconstruction fidelity of SFH time series using different methods, clearly illustrating the advantage of Kernel-PCA advantage in our specific application context.
• Fourth, we normalize the input features (the columns from X) via log-scaling, and follow this up by standard scaling normalization. In Figure 2, we visualize the log-scaled flux densities, in units of Janskies, for all three simulations, in 3 out of 20 filters.
• Finally, we derive KLIEP weights for the training samples in all three experiments. The Kernelized Learning for Independence (KLIEP) algorithm [16] is employed to adjust the weights of the training samples to minimize the Kullback–Leibler (KL) divergence between the source and target distributions. This instance-based domain adaptation approach is chosen due to its robustness in handling regression domain adaptation challenges and its effectiveness in mitigating negative transfer [2,13,63,64].

In our implementation, we carefully select the kernel and bandwidth parameters for KLIEP. The kernel type (Gaussian) and the bandwidth are crucial hyperparameters in this context. We conduct a grid search over a range of bandwidth values to identify the optimal setting that minimizes the KL divergence between the adapted source domain and the target domain. This process is detailed in Appendix B, where we showcase the parameter selection procedure for one of the experiments, demonstrating how the bandwidth impacts the effectiveness of domain adaptation.

Additionally, for the kernelPCA component, we explore a range of hyperparameters, specifically the kernel type and the $\gamma$ parameter in the Gaussian kernel, to find the best combination that allows for the most accurate reconstruction of the SFH time series. While Figure A1 illustrates examples with $\gamma$ values of 1 and 3, a broader range of values were examined to ensure robustness in the selection process. The effectiveness of each parameter setting is evaluated using the DILATE loss metric, providing a quantitative measure of the reconstruction’s fidelity compared to the original SFH.

We also visually observe from the marginal distributions for the 20 input filters that all domains have the same support in the feature space. In other words, we ensure that the distributions of input features across the different domains (simulations) cover similar ranges. This is crucial for the application of KLIEP, as the method assumes that the support of the source and target distributions overlap. To visually assess this, we examine the marginal distributions of the 20 input filters across the three simulations. If the distributions of each feature across the simulations show considerable overlap, we consider the support to be “the same” for practical purposes. This overlap ensures that the reweighing process by KLIEP is meaningful and grounded in comparable data ranges across domains. A detailed illustration of this overlap is provided in Appendix B and Figure A2, where kernel density estimates or histograms of feature distributions from different simulations are juxtaposed to highlight their common support.

For each of SFH_kPCA and SFH_sum, we employ an ensemble of 50 4-layer feed-forward deep neural networks with dropout and 256 nodes in each layer, using ReLU as the activation function and Adam [65] as the optimizer with a learning rate of $1e^{-3}$. The networks are trained for 200 epochs with early stopping to prevent overfitting. Employing multiple DNNs allows us to derive probability distributions over the SFH predictions, offering a nuanced understanding of the model’s performance and the uncertainty in its predictions. The choice of a 4-layer feed-forward DNN with 256 nodes per layer is guided by a balance between model complexity and computational efficiency, as well as empirical validation. The architecture is deep enough to capture the complex relationships between input features and the target SFHs, yet not so large as to risk overfitting given the available data size. The number of nodes is selected to provide sufficient capacity for learning intricate patterns without unnecessarily increasing the parameter count, which could lead to longer training times and require more data to avoid overfitting. This configuration is empirically tested against other architectures with varying depths and widths. The chosen setup consistently offers a good trade-off between learning capability and generalization performance, evidenced by the validation loss metrics and the model’s ability to predict SFHs on unseen data from the test simulation.

4. Results

The evaluation of the SFH outputs generated by our machine learning model is conducted through two primary methods:

• The SFH predictions for individual galaxies within the test set are compared against their corresponding true SFHs. This direct comparison provides a granular assessment of the model’s predictive accuracy on a per-galaxy basis.
• The aggregate predicted star formation histories $\Sigma$SFH, representing the sum of SFHs across all galaxies within a simulation, are compared to the true $\Sigma$SFHs. The true $\Sigma$SFH serves as an essential benchmark, reflecting the cumulative star formation activity and validating the simulation’s input physics. Observational evidence indicates that the universe’s star formation peaked approximately 2 billion years after its inception [55], a critical detail that all hydrodynamical simulations must replicate to be considered accurate. Ensuring concordance between the $\Sigma$SFH derived from our model and that from established simulations underpins the robustness of our approach. This alignment not only corroborates our model’s efficacy but also facilitates the fine-tuning of its architecture, loss functions, and hyperparameters in response to any discrepancies. Additionally, this comparison sheds light on potential systematic biases when the model is trained on one simulation and tested on another.

In Figure 3, we plot the true and derived $\Sigma$SFH curves for the three distinct test datasets (Simba, IllustrisTNG, and Eagle), where the training data consist of samples from the other two simulations. In these cases, the resulting sample SFHs (and thus $\Sigma$SFH) are most deviant from their ground truth vectors. This is unsurprising, as the different simulations have intrinsically different sample SFHs; this is a known difference between different simulations (as shown in Figure A1 of [66]).

Table 1. Forecasting results for individual SFHs. The first column gives the target domain IllustrisTNG, Eagle or Simba, and the two other domains are used as source domains. The computed metrics between each individual SFH prediction and the corresponding ground truth are averaged in the target domain. The $16\mathrm{th}$, $50\mathrm{th}$, and $84\mathrm{th}$ quantile values are drawn from the 50 predictions per galaxy, corresponding to the 50 neural networks used. The leading method between UDA and no-UDA (baseline) is shown in bold. Downward arrows for metrics imply that lower values are better.

Test Simulation		RMSE (↓)	MAE (↓)	BE (↓)	DTW (↓)	TDI (↓)
illustristng	Baseline	$0.3\begin{array}{c}+0.03\\ -0.01\end{array}$	$0.23\begin{array}{c}+0.02\\ -0.01\end{array}$	$0.04\begin{array}{c}+0.05\\ -0.04\end{array}$	$1.0\begin{array}{c}+0.08\\ -0.03\end{array}$	$5.13\begin{array}{c}+0.87\\ -0.89\end{array}$
	UDA	$\mathbf{0}.\mathbf{27}\begin{array}{c}+0.01\\ -0.01\end{array}$	$\mathbf{0}.\mathbf{2}\begin{array}{c}+0.01\\ -0.01\end{array}$	$\mathbf{0}.\mathbf{02}\begin{array}{c}+0.03\\ -0.02\end{array}$	$\mathbf{0}.\mathbf{94}\begin{array}{c}+0.02\\ -0.02\end{array}$	$\mathbf{3}.\mathbf{19}\begin{array}{c}+0.82\\ -0.41\end{array}$
eagle	Baseline	$0.43\begin{array}{c}+0.01\\ -0.01\end{array}$	$0.3\begin{array}{c}+0.01\\ -0.01\end{array}$	$-0.08\begin{array}{c}+0.01\\ -0.02\end{array}$	$1.59\begin{array}{c}+0.09\\ -0.07\end{array}$	$2.91\begin{array}{c}+2.49\\ -0.64\end{array}$
	UDA	$0.43\begin{array}{c}+0.01\\ -0.01\end{array}$	$0.3\begin{array}{c}+0.01\\ -0.01\end{array}$	$-0.09\begin{array}{c}+0.01\\ -0.01\end{array}$	$1.58\begin{array}{c}+0.07\\ -0.05\end{array}$	$2.21\begin{array}{c}+1.08\\ -0.36\end{array}$
	Baseline	$0.93\begin{array}{c}+0.03\\ -0.02\end{array}$	$0.63\begin{array}{c}+0.03\\ -0.02\end{array}$	$0.09\begin{array}{c}+0.05\\ -0.03\end{array}$	$3.14\begin{array}{c}+0.13\\ -0.11\end{array}$	$4.6\begin{array}{c}+0.38\\ -0.94\end{array}$
simba	UDA	$0.92\begin{array}{c}+0.02\\ -0.02\end{array}$	$0.6\begin{array}{c}+0.02\\ -0.02\end{array}$	$0.02\begin{array}{c}+0.03\\ -0.02\end{array}$	$3.25\begin{array}{c}+0.1\\ -0.12\end{array}$	$3.63\begin{array}{c}+0.88\\ -0.95\end{array}$

and

Table 2. Forecasting results for $\Sigma$SFH (total star formation history). The first column gives the target domain IllustrisTNG, Eagle or Simba, and the two other domains are used as source domains. Predictions for all samples in the target domain are summed and compared to the true $\Sigma$SFH according to the different metrics. The $16\mathrm{th}$, $50\mathrm{th}$, and $84\mathrm{th}$ quantile values are drawn from the 50 predictions per galaxy, corresponding to the 50 neural networks used. The leading method between UDA and no-UDA (baseline) is shown in bold. Metrics are scaled (MAE × 1000, RMSE × 1000, BE × 1000, DTW × 1000 for readability. Downward arrows for metrics imply that lower values are better.

Test Simulation		RMSE (↓)	MAE (↓)	BE (↓)	DTW (↓)	TDI (↓)
	Baseline	$0.72\begin{array}{c}+0.34\\ -0.13\end{array}$	$0.61\begin{array}{c}+0.26\\ -0.17\end{array}$	$0.34\begin{array}{c}+0.49\\ -0.34\end{array}$	$2.64\begin{array}{c}+1.37\\ -1.08\end{array}$	$0.17\begin{array}{c}+0.37\\ -0.08\end{array}$
illustristng	UDA	$\mathbf{0}.\mathbf{63}\begin{array}{c}+0.2\\ -0.15\end{array}$	$\mathbf{0}.\mathbf{49}\begin{array}{c}+0.15\\ -0.13\end{array}$	$\mathbf{0}.\mathbf{21}\begin{array}{c}+0.25\\ -0.18\end{array}$	$\mathbf{2}.\mathbf{09}\begin{array}{c}+1.17\\ -0.64\end{array}$	$\mathbf{0}.\mathbf{12}\begin{array}{c}+0.08\\ -0.05\end{array}$
	Baseline	$1.16\begin{array}{c}+0.1\\ -0.07\end{array}$	$0.84\begin{array}{c}+0.11\\ -0.06\end{array}$	$-0.4\begin{array}{c}+0.05\\ -0.08\end{array}$	$3.58\begin{array}{c}+0.48\\ -0.5\end{array}$	$0.47\begin{array}{c}+0.09\\ -0.11\end{array}$
eagle	UDA	$1.15\begin{array}{c}+0.08\\ -0.08\end{array}$	$0.85\begin{array}{c}+0.07\\ -0.09\end{array}$	$-0.4\begin{array}{c}+0.05\\ -0.04\end{array}$	$3.52\begin{array}{c}+0.36\\ -0.32\end{array}$	$0.47\begin{array}{c}+0.17\\ -0.15\end{array}$
	Baseline	$0.38\begin{array}{c}+0.09\\ -0.07\end{array}$	$0.3\begin{array}{c}+0.06\\ -0.05\end{array}$	$0.14\begin{array}{c}+0.09\\ -0.04\end{array}$	$1.17\begin{array}{c}+0.39\\ -0.27\end{array}$	$0.29\begin{array}{c}+0.1\\ -0.07\end{array}$
simba	UDA	$0.34\begin{array}{c}+0.05\\ -0.08\end{array}$	$0.29\begin{array}{c}+0.05\\ -0.08\end{array}$	$0.04\begin{array}{c}+0.04\\ -0.04\end{array}$	$0.82\begin{array}{c}+0.22\\ -0.14\end{array}$	$0.34\begin{array}{c}+0.16\\ -0.13\end{array}$

show the five metrics tested within this work —mean absolute error (MAE), root mean squared error (RMSE), bias error (BE), dynamic time warping (DTW), and temporal distortion index (TDI)—for average predictions of SHF, and for $\Sigma$SFH, respectively. These metrics are chosen for their ability to capture different aspects of the model’s accuracy and reliability in predicting SFHs. For instance, MAE and RMSE provide insights into the average prediction errors, while DTW and TDI offer a nuanced view of the model’s ability to capture the temporal dynamics of SFHs. Figure A4, Figure A5 and Figure A6 show examples of true and derived SFHs for individual galaxies within the simulations using KLIEP. Based on the five metrics, each row shows an example of the best-performing galaxy output on the left and the worst-performing galaxy on the right. It is noteworthy that the best and worst performers vary across different metrics, underscoring the distinct ways in which each metric evaluates time-series data. Consequently, while it is challenging to designate a single metric as the definitive loss function for minimization, RMSE is selected as the primary metric for several figures within this paper, facilitating a high-level understanding of the model’s performance.

Another observation made from these three figures is how well our technique can reproduce the stochastic nature of the simulated galaxies’ SFHs. As star formation can be an incredibly stochastic process (as is clear from examples such as the top-left panel in Figure A4 and the top-left panel in Figure A5), SFHs can regularly fluctuate between high and low values. In general, we find that such stochastic SFHs are poorly recovered. For the sake of galaxy property analysis, the accurate recovery of overall SFH trends is more important than the recovery of individual star formation rate (SFR) epochs. In the bottom-right panel of Figure A4, for example (galaxy index 333), it can be seen that there is a star formation event early on at ∼10–12 Gyr, and then a secondary star formation event from ∼2 Gyr to the present day. Recovering these two main epochs is more crucial than correctly recovering the individual SFR peaks within each epoch. One way of achieving this is to temporally smooth the SFHs of individual galaxies prior to training and testing. We can justify such smoothing of simulated features given that we would never expect the derived SFHs for observed galaxies (the ultimate aim of this work) to reproduce such short-scale stochastic features. As an example of this, see Figure A4 of Robotham et al. [18], where “good” fits to the SFHs from the semi-analytic model Shark [67] from the SED-fitting code ProSpect do not recover the stochastic SFHs.

To further contextualize our findings, we compare KLIEP with other popular domain adaptation methods3 in Appendix C—Margin Disparity Discrepancy (MDD) [68], Discriminative Adversarial Neural Network (DANN) [69], Kernel Mean Matching (KMM) [70], Correlation Alignment (CORAL) [71], and Deep Correlation Alignment (DeepCORAL) [72]. KMM, like KLIEP, is also an instance-based method, while the rest are feature-based methods. This comparison aims to highlight the strengths and potential areas for enhancement of our domain adaptation technique relative to other strategies. The curious reader will notice that while at times CORAL methods outperform KLIEP, this is very much a data-dependent result, and KLIEP provides the most reliably acceptable results across datasets, making it more generalizable. This is most clearly apparent in Figure A3.

5. Future Work

The results presented here are only one part of a continuing effort to apply sophisticated data-driven methods to SED fitting, a crucial first step for extracting galaxy properties. In future work, we will explore advanced multi-source domain adaptation approaches [73,74], which should more robustly account for the presence of multiple source-domain datasets as well as adaptively reweigh the training samples depending on the prediction task, thus providing higher quality results.

We are also actively analyzing and attempting to correct systematic biases induced by our various design choices. How do our SFH inferences for galaxies of low mass compare to those of high mass? Is our modeling sufficient to accurately infer SFH from both old and young galaxies, unlike traditional parametric approaches, which suffer at earlier epochs in the universe’s history? Answers to such questions will enable us to discover and correct potential biases in our predictions.

Finally, we also plan on vastly increasing the size of our simulated data to account for the immense diversity in the physics of galaxy formation and evolution modeling–different initial mass functions (IMFs) (for example [75,76,77]), nebular emission models (for example [78]), and mass–metallicity relationships (such as those presented by [79,80,81]), among others. While this task has a relatively longer horizon—generating and saving new simulations is a compute-intensive task—it is crucial to undertake before data-driven methods such as the one proposed in this paper become trusted by astronomers.