Red Noise Suppression in Pulsar Timing Array Data Using Adaptive Splines

Abstract

Noise in Pulsar Timing Array (PTA) data is commonly modeled as a mixture of white and red noise components. While the former is related to the receivers, and easily characterized by three parameters (EFAC, EQUAD and ECORR), the latter arises from a mix of hard to model sources and, potentially, a stochastic gravitational wave background (GWB). Since their frequency ranges overlap, GWB search methods must model the non-GWB red noise component in PTA data explicitly, typically as a set of mutually independent Gaussian stationary processes having power-law power spectral densities. However, in searches for continuous wave (CW) signals from resolvable sources, the red noise is simply a component that must be filtered out, either explicitly or implicitly (via the definition of the matched filtering inner product). Due to the technical difficulties associated with irregular sampling, CW searches have generally used implicit filtering with the same power law model as GWB searches. This creates the data analysis burden of fitting the power-law parameters, which increase in number with the size of the PTA and hamper the scaling up of CW searches to large PTAs. Here, we present an explicit filtering approach that overcomes the technical issues associated with irregular sampling. The method uses adaptive splines, where the spline knots are included in the fitted model. Besides illustrating its application on real data, the effectiveness of this approach is investigated on synthetic data that has the same red noise characteristics as the NANOGrav 15-year dataset and contains a single non-evolving CW signal.

1. Introduction

Pulsar Timing Arrays (PTAs) aim to detect the gravitational waves (GWs) at nano-hertz frequencies by precisely monitoring an ensemble of highly stable millisecond pulsars (MSPs) distributed across the galaxy. Among the expected sources of GWs in this frequency range are individual Supermassive Black Hole Binaries (SMBHBs) [1] and the stochastic GW background (GWB) from the entire population of SMBHBs. The fundamental methodology begins with the collection of pulse times of arrival (ToAs) from these pulsars over many years. For each pulsar, a detailed deterministic timing model is constructed, accounting for factors such as its spin-down, binary motion (if applicable), astrometric parameters, and interstellar medium effects [2,3]. The differences between the observed ToAs and the predictions from this timing model are known as timing residuals. These residuals are then carefully analyzed to characterize and mitigate various noise sources, including the frequency-dependent red noise intrinsic to each pulsar and white noise associated with measurement uncertainties [4]. The GWB is anticipated to manifest as an additional stochastic process that is common to all pulsars in the array but exhibits a specific quadrupolar spatial correlation pattern. This distinctive inter-pulsar correlation is described by the Hellings and Downs (HD) curve [5,6,7,8], which predicts the expected correlation coefficient as a function of the angular separation between any pair of pulsars for an isotropic, unpolarized GWB.

Statistical frameworks, predominantly Bayesian inference, are employed to analyze the ensemble of timing residuals. This involves constructing a likelihood function that models the probability of the observed data given a set of parameters describing individual pulsar noise, any common uncorrelated red noise, and the GWB signal (typically parameterized by its amplitude and spectral index) [9,10,11,12,13,14,15,16]. Techniques like Parallel Tempering Markov Chain Monte Carlo (PTMCMC) [17] are often used to explore this parameter space, derive posterior probability distributions for the GWB parameters, and compute Bayes factors to evaluate the statistical evidence for a GWB signal compared to models without it. While the specifics of noise modeling and data handling may differ slightly among PTA collaborations [18], this general approach of analyzing timing residuals, characterizing noise, and searching for the characteristic HD spatial correlations forms the common backbone of GWB detection and characterization efforts.

Recently, PTAs have made significant strides in characterizing the GWB. While early analyses from NANOGrav’s 12.5-year dataset [9] and PPTA DR2 [11] identified a common-spectrum process, definitive HD spatial correlations [5] have remained elusive. Subsequently, multiple PTAs have all reported varying levels of evidence for an HD-correlated GWB. These include ∼4$\sigma$ findings from NANOGrav with its 15-year dataset [10] (henceforth the NG15 dataset) and PPTA [13], a ∼3–4$\sigma$ signal from EPTA in conjunction with InPTA [12], a 4.6$\sigma$ HD-correlated signal from CPTA [14], and 3–3.4$\sigma$ evidence from MPTA [15,16]. Despite differing modeling approaches, the GWB parameters measured by these collaborations are largely consistent [18], with the signal predominantly interpreted as arising from supermassive black hole binary mergers (SMBHBs) [1,19].

Several searches for individual SMBHBs have been carried out [20,21,22,23], but a statistically significant detection is yet to be made. Signals from isolated SMBHBs in the PTA band are often called continuous wave (CW) signals due to their long lifetimes that far exceed observational time scales. The statistical problem posed by CW detection differs in a crucial aspect from GWB searches. For CWs, the signal is deterministic and coherent across the PTA, and the primary objective is to identify this specific waveform and sky location. Consequently, unlike GWB analysis, where distinguishing the power spectral density (PSD) of the GWB from intrinsic red noise PSDs is paramount, CW searches benefit from treating both intrinsic red noise and any underlying GWB as a combined noise background that needs to be effectively mitigated or filtered out. Current Bayesian approaches to CW detection often extend the already complex GWB search models by incorporating additional parameters for the CW signal [24,25], leading to an even higher dimensional parameter space and further intensifying the computational burden.

To circumvent this, we propose a novel methodology specifically tailored for CW searches. This approach introduces an explicit technique for filtering out the combined red noise and GWB background, thereby significantly reducing the complexity of the subsequent search for CW signals and offering a computationally efficient pathway to their detection. Our method uses an algorithm called SHAPES (Swarm Heuristics-based Adaptive and Penalized Estimation of Splines), which utilizes an adaptive spline curve fitting method [26], to model and subtract red noise from the data of each pulsar. The SHAPES-based pipeline is computational efficient and scalable, making it well-suited for large PTA datasets.We illustrate the method on NANOGrav data as well as simulated PTA data. With the latter, we conduct a search for an injected signal and demonstrate that the red noise suppression method does not adversely affect the detectability of signals outside the the frequencies dominated by the red noise. Notably, the entire pipeline completed in a few hours, representing a significant reduction in computational cost compared to Bayesian analyses on the same dataset, which typically require tens of hours or more on the same computing platform.

The rest of the paper is organized as follows: in Section 2, we describe the methods used in this work, including the adaptive spline fitting technique for red noise removal. In Section 3, we present the results of our simulations and the performance of our red noise removal algorithm in a single CW search. Finally, in Section 4, we discuss the implications of our findings and potential future work.

2. Methods

In the Bayesian framework, red noise is commonly modeled as a power law [4,20,21,23,24,27,28,29]. This introduces two free parameters for each pulsar—the spectral index (RNIDX) and amplitude (RNAMP)—that must be fitted, thus increasing the computational burden, which scales with the number of pulsars in the PTA. However, in a search for a CW signal under a Bayesian or Likelihood-based Fisherian formalism, the search statistic is composed of inner products of the data with template waveforms that are weighted by the reciprocal of the PSD. This means that the search statistic essentially suppresses the frequencies where the red noise dominates. This situation is similar to that of LIGO data analysis where the red noise originates from ground motion. As in the case of LIGO data analysis, one can take the approach of explicitly filtering out the red noise from PTA data, thereby alleviating the need to fit the red noise parameters. The principal technical problem in implementing this for PTA data analysis is that it is irregularly sampled, unlike LIGO data.

To handle the explicit filtering of red noise from irregularly sampled data, we introduce a novel time-domain approach based on using a method called SHAPES (Swarm Heuristics-based Adaptive and Penalized Estimation of Splines). SHAPES is an implementation in MATLAB of the adaptive spline fitting algorithm described in [26], and has been successfully applied in GW data analysis for glitch subtraction in LIGO data [30]. Operating directly on the residual data, SHAPES models and subtracts red noise without the need for frequency-domain parameterization. The SHAPES method fits data using an adaptive spline, where the knot locations are optimized via Particle Swarm Optimization (PSO) [31,32]. For any given set of knots, the spline is formed as a linear combination of B-spline basis functions [33]. The values of these B-splines are calculated using the de Boor recursion relations, which are inherently unbiased and function correctly with both regularly and irregularly spaced data. However, the current implementation of SHAPES is limited to regularly sampled data. This is because it employs a computationally efficient version of the recursion relations that leverages symmetries only present in uniform sampling. Future work will involve adopting the native form of the de Boor recursions to support irregularly sampled data. To offset the anticipated loss in computational efficiency from this change, the implementation will be migrated from its current MATLAB environment to a compiled language such as C.

For making this paper self-contained, we provide a brief overview of the essential ingredients of SHAPES and refer the reader to [26,30] for the details. The fundamental underlying model for SHAPES posits that the observed data $\overline{y}$ is a composite of a true signal $\overline{s}\left(\theta \right)$, with parameters $\theta$ and additive white noise $\overline{ϵ}$, mathematically represented as

$$\overline{y}=\overline{s}\left(\theta \right)+\overline{ϵ}.$$

(1)

In this formulation, $\overline{y}$, $\overline{s}\left(\theta \right)$, and $\overline{ϵ}$ are N-element row vectors, where ${\overline{y}}_i=\overline{y}\left(t_i\right)$ and ${\overline{s}}_i\left(\theta \right)=\overline{s}(t_i;\theta )$ are samples at discrete time points $t_i$ (that need not be regularly spaced). Within the SHAPES framework, the signal $\overline{s}(t;\theta )$ is specifically modeled as a cubic spline, which is a piecewise polynomial of order 4, and can be represented by a linear combination of B-spline functions [33]

$$s(t;\theta =\{\overline{\alpha },\overline{\tau }\})=\sum _{j=0}^{P-5}{\alpha }_j{B}_{j,4}(t;\overline{\tau }),$$

(2)

where $B_{j,4}(t;\overline{\tau })$ denotes the B-spline basis functions [30], $\overline{\tau }$ is the vector specifying the knot locations (knots are the boundaries of the polynomial pieces in a spline), and $\overline{\alpha }$ are independent coefficients in the linear combination of the B-splines above. The determination of the knot locations $\overline{\tau }$, which are a crucial component of $\theta$, for a given number of knots P, is accomplished using a local-best variant of PSO metaheuristic. To ascertain the optimal number of knots P, SHAPES employs model selection guided by the Akaike Information Criterion (AIC) [34], which aims to find a balance between the model’s goodness-of-fit and its complexity (i.e., the number of knots). For practical implementation, the algorithm typically minimizes the AIC over a finite set of knot numbers. The best-fit parameters $\overline{\alpha }$ and $\overline{\tau }$ are those that minimize a penalized least-squares function:

$$\begin{array}{c}{L}_{\lambda }(\overline{\alpha },\overline{\tau })=L(\overline{\alpha },\overline{\tau })+\lambda R\left(\overline{\alpha }\right),\\ L(\overline{\alpha },\overline{\tau })={\displaystyle \sum _{i=0}^{N-1}}{\left(y_i-s_i(\overline{\alpha },\overline{\tau })\right)}^2,\end{array}$$

(3)

with the penalty term defined as

$$R\left(\overline{\alpha }\right)=\sum _{j=0}^{P-5}{\alpha }_j^2,$$

(4)

The user-specified parameter $\lambda$, called the penalty gain, governs the smoothness of the resulting spline estimate. A key feature of SHAPES is that knots can be collapsed to have higher multiplicity, allowing discontinuities to be fitted along with the smooth parts automatically.

The procedure for red noise removal from the pulsar timing residuals is detailed in the following steps:

• Uniform Sampling and Initial Estimation: For each pulsar, the irregularly sampled timing residuals are initially converted into a uniformly sampled sequence via interpolation based on their observation epochs. This uniform sequence is then processed using the SHAPES algorithm to obtain a preliminary estimate of the residuals.
• Reversion to Original Cadence: The estimates generated by SHAPES on the uniform grid are interpolated back to the original, non-uniform observation epochs of each pulsar. This step yields estimates that align precisely with the original data points.
• Low-Pass Filtering: The resulting SHAPES estimates is low-pass filtered to isolate the low-frequency noise components. This is accomplished using a zero-phase digital filter implemented with the filtfilt function in MATLAB, which ensures that the filtering process introduces no phase distortion. A cutoff frequency of $2\times {10}^{-8}$ Hz is utilized. The order of the filter is determined for each pulsar separately by the formula $\mathtt{floor}(\mathrm{nSample}/\mathrm{R}-1)$, where $\mathrm{nSample}$ is the number of data points and the hyperparameter $\mathrm{R}$ is introduced to satisfy the minimum data length requirement of the filter. An optimal value of $\mathrm{R}=8$ was selected empirically for this study.
• Noise Subtraction: In the final step, the low-pass filtered SHAPES estimates, representing the extracted red noise, are subtracted from the original timing residuals.

The performance of the SHAPES algorithm was evaluated using observational data from the NG15 dataset for pulsar PSR J1909-3744. The efficacy of this method in suppressing intrinsic red noise is illustrated in Figure 1, which depicts the original NG15 data, the SHAPES estimates, and the resulting timing residuals. The data was processed with a Finite Impulse Response (FIR) low-pass filter with a cutoff frequency at 20 nHz. This specific value was chosen strategically for the purposes of this proof-of-concept study. Our primary objective here was to validate the algorithm’s performance by recovering a simulated CW signal injected into the data at a known frequency of 26 nHz. Therefore, selecting a cutoff at 20 nHz ensures that the filter characterizes the red noise without attenuating the target signal of interest. We acknowledge that a single, fixed cutoff frequency is suboptimal for a blind search in real observational data where signal parameters are unknown a priori. In such a scenario, a more sophisticated, data-driven approach is required. The optimal cutoff frequency should be determined adaptively for each pulsar, likely based on an analysis of its PSD. The development of such an adaptive filtering strategy is a key objective for our future work involving searches in real pulsar timing data. Figure 2 shows the Lomb–Scargle Periodogram (LSP) [35] of the data and residual after subtraction of the red noise estimate. It corroborates the effectiveness of this approach, as the power of the red noise below $3\times {10}^{-9}$ Hz is suppressed by approximately three orders of magnitude.

3. Results

We use PINT, which is an open-source Python alternative to the pulsar timing package TEMPO2 [36,37,38], to simulate realistic timing residuals. The simulations are based on the NG15 data [39], which includes observations of 68 pulsars over a span of more than 15 years. Among these pulsars, 23 were found to have detectable levels of red noise. While most of the 68 pulsars are observed at an approximately monthly cadence, 6 of the highest-precision pulsars benefit from additional high-cadence observations performed roughly every five days. The released data is composed of two parts: (1) the observed ToAs for each pulsar, stored in files that are called tim files, and (2) the timing model for each pulsar, stored in files called par files. Here, we use the par files from the released data in PINT to obtain simulated realizations of timing residual noise for each pulsr. The simulation includes the following steps:

• Within the observation duration for each pulsar as given in its par file, create a sequence of regularly spaced observation epochs with 14 days cadence. This is followed by the addition of a 3-day random shift to each epoch to form a sequence of irregularly sampled epochs. The random shift is drawn from an uniform distribution.
• Using PINT, a noise realization for each pulsar is generated using its irregularly spaced sequence of observation epochs. For this, we set the timing uncertainty (i.e., the white noise standard deviation) for all pulsars to 100 ns, and set the intrinsic red noise PSD according to each pulsar’s par file.
• A CW signal, as described below, is injected into the simulated noise.

We consider the simplest type of a CW signal, namely, one that is emitted by an unevolving and circular orbit SMBHBs. The timing residual $S^I(t;\theta )$ induced by such a source for the I’th pulsar in the PTA is given by [40]

$$s^I(t;\theta )={\int }_0^t\mathrm{d}{t}^{\prime }{z}^I(t^{\prime };\theta ),$$

(5)

where $z^I(t;\theta )\equiv ({\nu }^I(t;\theta )-{\nu }_0^I)/{\nu }_0^I$ is the GW-induced Doppler shift, given by the difference between the frequency of the radio pulses observed at the Solar System Barycenter (SSB), ${\nu }^I(t;\theta )$ and the rest frame of the pulsar, ${\nu }_0^I$. For a plane GW traveling across the Earth-pulsar line of sight, originating from a source positioned in equatorial coordinates with right ascension $\alpha$ and declination $\delta$, the function $z(t;\theta )$ can be represented as

$$z^I(t;\theta )=\sum _{A=+,\times }{F}_A^I(\alpha ,\delta )\Delta h_A(t;{\theta }_s),$$

(6)

where $\theta =\{\alpha ,\delta \}\cup {\theta }_s$ and $F_{+,\times }^I(\alpha ,\delta )$ are the antenna pattern functions for the + and × polarizations of the GW [41,42]. The two–pulse response [43],

$$\Delta h_{+,\times }(t;{\theta }_s)=h_{+,\times }(t;{\theta }_s)-h_{+,\times }(t-{\kappa }^I;{\theta }_s).$$

(7)

It includes the so-called Earth and pulsar terms that result from the influence of gravitational waves on the pulses at the times t and $t-{\kappa }^I$, corresponding to their reception and emission, respectively. The time delay ${\kappa }^I$ can be considered an unknown constant phase offset, referred to as the pulsar phase parameter ${\varphi }_I$, for a non-evolving source.

The injected signal in our simulation has monochromatic $h_{+}$ and $h_x$ with an unevolving frequency of $2.63\times {10}^{-8}$ Hz and a network SNR around 57. Here, the network SNR is defined as

$$\begin{array}{c}\mathrm{SNR}={\left[{\displaystyle \sum _{I=1}^{N_p}}{\left({\rho }^I\right)}^2\right]}^{1/2},\\ {\rho }^I={\left({\mathbf{r}}^I\right)}^T·{\mathbf{C}}^{-1}·{\mathbf{r}}^I,\end{array}$$

(8)

where ${\rho }^I$ and ${\mathbf{r}}^I$ are the per-pulsar SNR and timing residuals for Ith pulsar, and C is the noise covariance matrix.

The NG15 data release encompasses timing observations of 68 pulsars, and its total observational baseline spans 15 years, measured from the earliest to the latest observation time across the entire pulsar array. Among these pulsars, 23 exhibit statistically significant red noise. Taking PSR J1909-3744 as an example, Figure 3 shows the simulated timing residuals in which the injected CW signal described above is added to the PINT simulated noise. We also analyze the timing residuals using the LSP as shown in Figure 4. This clearly shows the very strong red noise power in the low frequency range $\lesssim 4\times {10}^{-9}$ Hz, and the peak arising from the injected signal at $2.63\times {10}^{-8}$ Hz.

Figure 5 shows the LSP for the injected CW signal, the combined residuals, and the residuals obtained after subtracting the SHAPES estimates from the combined residuals, both with and without low-pass filtering. It demonstrates that subtracting the SHAPES estimates without low-pass filtering may result in a loss of power in the injected signal. If we first low-pass filter the SHAPES estimates with a cutoff frequency slightly below that of the injected CW signal, and then subtract it, we can preserve most of the signal’s power while still achieving effective noise suppression at lower frequencies.

Having prepared the red noise suppressed residuals as described above, we use an algorithm introduced in [42,44,45,46] with a pulsar selection scheme [47] to perform a single source search in our simulated dataset. Here, we briefly review the essential features of the search algorithm. The algorithm uses a likelihood-based approach that fully accounts for both the Earth and pulsar terms in the signal model. For a Pulsar Timing Array, the timing residual data for the I-th pulsar is modeled as ${\overline{y}}_I={\overline{s}}_I\left(\theta \right)+{\overline{n}}_I+{\overline{e}}_I$, where ${\overline{s}}_I\left(\theta \right)$ is the GW-induced signal characterized by the set of parameters $\theta$ and ${\overline{n}}_I$ is a realization of the noise associated with the ToAs, while ${\overline{e}}_I$ is the error rising from fitting ToAs to the timing model, and it is typically expressed as ${\overline{e}}_I={\mathbf{M}}_I\delta p_I$, where ${\mathbf{M}}_I$ is the design matrix used in the fitting procedure, and $\delta p_I$ represents the differences between the best-fit and the true parameters [37]. The source parameters $\theta$ are divided into 7 intrinsic parameters (GW frequency, sky location (RA, DEC), GW amplitude, inclination angle, GW polarization angle and orbital initial phase) and extrinsic parameters, which are the unknown pulsar phase parameters ${\varphi }_I$ for each pulsar. To efficiently handle these parameters, we utilizes a hybrid method called MaxAvPhase, which combines two complementary algorithms: AvPhase and MaxPhase. First, AvPhase is employed to estimate the intrinsic parameters by maximizing a marginalized likelihood, where the pulsar phases ${\varphi }_I$ have been integrated out. This step is guided by the Marginalized-Maximized Likelihood Ratio Test (MMLRT) statistic, defined as

$$\mathrm{MMLRT}\left(Y\right)=\underset{{\lambda }_i}{max}ln({\mathrm{marg}}_{{\lambda }_e}\Lambda (Y;\lambda )).$$

Subsequently, with the intrinsic parameters fixed to the values found by AvPhase, the extrinsic parameter estimation step of MaxPhase is applied to obtain point estimates for the pulsar phases ${\varphi }_I$. This combined strategy leverages the robust intrinsic parameter estimation of AvPhase, especially at lower SNR [46], while still providing the complete set of parameter estimates needed to reconstruct the signal waveform. In this study, we inject an SNR 57 CW signal with a frequency of $2.63\times {10}^{-8}$ Hz, very close to our chosen low-pass filter cutoff frequency of 20 nHz. The signal is detected successfully using our pipeline once the red noise is filtered out using our SHAPES-based method. In terms of parameter estimation, the search achieves a 29.05% relative SNR error and a 0.44% relative frequency error. The SHAPES code will be adapted to irregularly sampled data, thereby eliminating one non-essential step from our method.

4. Conclusions

In this work, we introduced a novel approach for mitigating red noise in individual pulsar ToAs, offering an alternative to computationally expensive Bayesian methods. Our technique employs adaptive spline curve fitting in the time domain to estimate and remove red noise without requiring explicit frequency-domain models or global fitting procedures. We demonstrated the efficacy of this method on the NG15 dataset for PSR J1909-3744 which shows a substantial suppression of the power of red noise below $3\times {10}^{-9}$ Hz and performed a CW search on a synthetic PTA data based on the NG15 data. Our pipeline successfully suppressed the red noise while preserving the signal, enabling its detection with a relative SNR error of 29.05% and a frequency error of 0.44%.

The computational efficiency of this framework is a key advantage. For context, the analysis presented in this paper required only several hours of processing time, a stark contrast to the 2–3 days typically necessary for a comparable analysis using traditional Bayesian methods. This dramatic speed-up shows significant promise for accelerating searches for continuous gravitational waves in large datasets.

Building on these results, we are preparing a comprehensive follow-up study dedicated to performance benchmarking. This subsequent paper will provide a detailed comparison of the method’s processing times against traditional Bayesian methods. It will also include a rigorous analysis of detection statistics to fully quantify the method’s reliability. While such an extensive analysis is beyond the scope of this introductory paper, we believe it is a crucial next step that will provide a valuable, quantitative resource for researchers at the forefront of gravitational wave astronomy.