Extending the Fully Bayesian Unfolding with Regularization Using a Combined Sampling Method

Abstract

Regularization extensions to the Fully Bayesian Unfolding are implemented and studied with an algorithm of combined sampling to find, in a reasonable computational time, an optimal value of the regularization strength parameter in order to obtain an unfolded result of a desired property, like smoothness. Three regularization conditions using the curvature, entropy and derivatives are applied, as a model example, to several simulated spectra of top-pair quark pairs that are produced in high energy $pp$ collisions. The existence of a minimum of a ${\chi }^2$ between the unfolded and particle-level spectra is discussed, with recommendations on the checks and validity of the usage of the regularization feature in Fully Bayesian Unfolding (FBU).

1. Introduction

This study is motivated by unstable results of unfolding in the case of spectra with a large number of bins or with a poorly diagonal migration matrix. In order to obtain the expected smooth results, a bias is carefully introduced via a regularization function. We focused on the Fully Bayesian Unfolding (FBU) method [1].

Other unfolding methods than FBU have already been introduced, including regularization features. Because FBU has the ability to provide the full probability posterior of the result and thus stands out from other methods, it is interesting to also provide FBU with the regularization option.

This study uses several over-binned spectra, so the unfolding without regularization results in fluctuating, high-variance unfolded spectra. These were constructed by fine binning, so that the migration matrix is under-represented on diagonal. Five typical spectra considering the production of top quark pairs in $pp$ collisions were chosen, namely the pseudorapidity ${\eta }^{\mathrm{t},\mathrm{had}}$ and transverse momentum $p_T^{\mathrm{t},\mathrm{had}}$ of the hadronicaly decaying top quark, and pseudorapidity ${\eta }^{\mathrm{t}\overline{\mathrm{t}}}$, transverse momentum $p_T^{\mathrm{t}\overline{\mathrm{t}}}$, and mass $m^{\mathrm{t}\overline{\mathrm{t}}}$ of the $t\overline{t}$ system.

The regularization feature was introduced as a prior in the likelihood. However, the problem which remains is how strongly should user apply the regularization to get the optimal unfolded spectrum, which should be close to the particle spectrum, but, at the same time, respects the desired regularization property, e.g., low curvature. The power of regularization is governed by the regularization strength parameter $\tau$. The higher the $\tau$, the stronger the regularization.

An algorithm for finding the optimal value of $\tau$ in a reasonable time via a combined sampling is given so that the best unfolding result is achieved, representing the main message of this paper.

2. Unfolding

The unfolding procedure corrects measured spectra for the resolution, efficiency, and acceptance effects of the detector usually to the particle level. The principles of the Fully Bayesian Unfolding method are based on the Bayes theorem

$$P\left(A\right|B)=\frac{P\left(B\right|A)·P(A)}{P\left(B\right)},$$

(1)

where the conditional probability $P\left(A\right|B)$ reads the probability of A, given as B. Fully Bayesian Unfolding provides the full probability posterior $P\left(A\right|B)$ of the result, which gives extra information when compared to other unfolding methods [2]. Rewriting Equation (1), the probability of the truth spectrum $\mathbf{T}$, given the data $\mathbf{D}$, is

$$P\left(\mathbf{T}\right|\mathbf{D})=\frac{P\left(\mathbf{D}\right|\mathbf{T})·\pi (\mathbf{T})}{\mathrm{Norm}.},$$

(2)

where the $\mathrm{Norm}.$ is a normalization factor, $P\left(\mathbf{D}\right|\mathbf{T})$ is the likelihood function, and $\pi \left(\mathbf{T}\right)$ is a prior probability density for $\mathbf{T}$. In the case of $\pi \left(\mathbf{T}\right)=1$ the algorithm is non-regularized. By inserting an arbitrary regularization function instead of $\pi \left(\mathbf{T}\right)$, one can intentionally bias the final result towards a suitable property, e.g., to be smooth, and unfolding is then called regularized. The convenient way is to use an exponential function with regularization strength parameter $\tau$ and an inner function $S\left(\mathbf{T}\right)$

$$P\left(\mathbf{T}\right|\mathbf{D})=\frac{P\left(\mathbf{D}\right|\mathbf{T})·e^{-\tau S\left(\mathbf{T}\right)}}{\mathrm{Norm}.}.$$

(3)

If the parameter $\tau =0$, the prior $\pi \left(\mathbf{T}\right)=1$, and no regularization is applied; on the other hand, the higher value of $\tau$, the more dominant the regularization term.

2.1. Simulated Spectra

The input pseudo-data spectrum ${\mathbf{D}}_{\mathrm{sim}}$ is simulated for the process $pp\to t\overline{t}$ at $\sqrt{s}=14\mathrm{TeV}$ while using MadGraph5 [3] software, Pythia8 generator [4], and Delphes [5] detector simulation with an ATLAS card while using basic kinematics cuts in order to obtain a realistic sample of top quark pairs decaying in a semi-leptonic way $t\overline{t}\to bWbW\to \ell +\mathrm{jets}$, with more details being described in [6].

2.2. Defining Equations of Unfolding

The necessary ingredients for unfolding are the data spectrum $\mathbf{D}$ and the migration matrix M, which is normalized, so that it maps the probability of an event at the truth (particle) level bin i to be reconstructed (measured) at the detector level bin j.

The schematic formula describing unfolding for the case of a simple migration matrix inversion is

$$\widehat{T_i}=\frac{1}{{ϵ}_i}·{\left(M^{-1}\right)}_{ij}·{\eta }_j·{(D-B)}_j.$$

(4)

In this paper, the background spectrum $\mathbf{B}$ is not taken into account. The efficiency and acceptance corrections are derived from projections of the migration matrix M divided by the simulated particle level $\mathbf{T}$ or data spectrum $\mathbf{D}$, as

$$ϵ=\frac{{\tilde{\mathbf{T}}}_{\mathrm{proj}.\mathrm{from}\mathrm{M}}}{\mathbf{T}};\eta =\frac{{\tilde{\mathbf{D}}}_{\mathrm{proj}.\mathrm{from}\mathrm{M}}}{\mathbf{D}}.$$

(5)

Because our data are simulated we set $\mathbf{D}$ = ${\mathbf{D}}_{\mathrm{sim}}$ and Equation (4) becomes

$$\widehat{\mathbf{T}}=\frac{1}{ϵ}·M^{-1}·\eta ·{\mathbf{D}}_{\mathrm{sim}}.$$

(6)

The production of spectra was divided into two statistically independent sets A and B, where set A is used for unfolding components and set B is taken as pseudo data ${\mathbf{D}}_{\mathrm{sim}\mathrm{B}}$ input to unfolding

$${\widehat{\mathbf{T}}}_B=\frac{1}{{ϵ}_A}·M_A^{-1}·{\eta }_A·\left({\mathbf{D}}_{\mathrm{sim}\mathrm{B}}\right).$$

(7)

The result of unfolding ${\widehat{\mathbf{T}}}_B$ is compared to the corresponding particle level spectrum ${\mathbf{T}}_B$ (green line in Figure 1a). The ${\chi }^2$ test between the unfolded and particle spectrum divided by the number of degrees of freedom (number of the bins), ${\chi }^2/\mathrm{ndf}$, is calculated in the standard way, as follows

$${\chi }^2/\mathrm{ndf}=\frac{1}{N}\sum _{i=1}^N\frac{{({\widehat{T}}_B^i-T_B^i)}^2}{{\sigma }_{{\widehat{T}}_B^i}^2+{\sigma }_{T_B^i}^2}.$$

(8)

The input components that are needed for the unfolding process are visualized for the case of the ${\eta }^{t\overline{t}}$ spectrum presented in Figure 1.

In essence, the FBU unfolding technique exploits the response matrix to fold a probe truth-level spectrum, obtaining the detector-level spectrum, and computing a likelihood between such folded spectrum and the actual data. This elegantly avoids issues of singularities in explicit matrix inversion or their regularization, as e.g., in the SVD technique [7]. The tricky part of the FBU is how to effectively sample the truth space in the region giving large likelihood values.

2.3. Likelihood Function $P\left(\mathbf{D}\right|\mathbf{T})$

The likelihood function is given by the product of Poisson distributions in case of counts as

$$\begin{array}{cc}\hfill & P\left(\mathbf{T}\right|\mathbf{D})\propto L\left(\mathbf{D}\right|\mathbf{T})·\pi \left(\mathbf{T}\right)=\left(\prod _{i=1}^N\frac{{\lambda }_i^{k_i}}{k_i!}{e}^{-{\lambda }_i}\right)e^{-{\tau }_{\mathrm{abs}}S\left(\mathbf{T}\right)},\hfill \end{array}$$

(9)

where

$${\lambda }_i=\sum _{j=1}^N{M}_{ij}{T}_j$$

(10)

and

$$k_i={\eta }_i{(D-B)}_i.$$

(11)

After unfolding, the probability $P\left(\mathbf{T}\right|\mathbf{D})$ is divided by the efficiency correction $ϵ$. The acceptance correction $\eta$ that is shown in Figure 1b takes on values [0,1] and, thus, the factorial is computed using the Gamma function as

$$x!=\mathsf{\Gamma }(x+1).$$

(12)

In the simplified case of no background ($\mathbf{B}=0$), pseudo data $\mathbf{D}={\mathbf{D}}_{\mathrm{sim}}$, and introducing the natural logarithm, since the sampling of the functions is more numerically stable, Equation (9) becomes

$$\begin{array}{cc}\hfill & logP\left(\mathbf{T}\right|{\mathbf{D}}_{\mathrm{sim}})\propto log(L\left({\mathbf{D}}_{\mathrm{sim}}\right|\mathbf{T})·\pi \left(\mathbf{T}\right))=\hfill \\ & =\sum _{i=1}^{N}log\left(\frac{{\lambda }_i^{\left[{\eta }_i{\left(D_{\mathrm{sim}}\right)}_i\right]}}{\left[{\eta }_i{\left(D_{\mathrm{sim}}\right)}_i\right]!}{e}^{-{\lambda }_i}\right)-{\tau }_{\mathrm{abs}}S\left(\mathbf{T}\right),\hfill \end{array}$$

(13)

which can be rewritten as

$$logP\left(\mathbf{T}\right|{\mathbf{D}}_{\mathrm{sim}})\propto logL\left(\mathbf{T}\right)-{\tau }_{\mathrm{abs}}S\left(\mathbf{T}\right).$$

(14)

This expression has the advantage of allowing one to find the maximum of $logP\left(\mathbf{T}\right|{\mathbf{D}}_{\mathrm{sim}})$ by sampling the log-likelihood function $logL\left(\mathbf{T}\right)$ and the regularization function $S\left(\mathbf{T}\right)$ separately and tune the regularization parameter $\tau$, so that the optimal $\tau$ can be found in a much shorter time than sampling the full likelihood for every possible value of $\tau$.

In this study, both of the approaches were used: the one when sampling was applied to the function $P\left(\mathbf{T}\right|{\mathbf{D}}_{\mathrm{sim}})$ as a whole, denoted as full sampling, and the one when sampling of the two components $logL\left(\mathbf{T}\right)$ and $S\left(\mathbf{T}\right)$ is performed separately, denoted as combined (fast) sampling. These methods will be described in detail in Section 2.6.

Figure 2a shows an example of the log-likelihood, sampling of the function $logL\left(\mathbf{T}\right)$ for the spectrum ${\eta }^{t\overline{t}}$ and its projection to the $6^{\mathrm{th}}$ and $9^{\mathrm{th}}$ bin $logL(T_6,T_9)$. Figure 2b is created while using the highest points of the $logL(T_6,T_9)$ and, thus, represents an envelope of the $logL(T_6,T_9)$, while Figure 2c represents the exponential of the envelope of the original $L(T_6,T_9)$, with the posteriors plotted along the x and y axes (red lines), which are the marginalized binned probability densities that are analytically given as

$$p_i\left(T_i\right|\mathbf{D})=\int \cdots \int P\left(\mathbf{T}\right|\mathbf{D})d{T}_1\dots d{T}_{i-1}d{T}_{i+1}\dots d{T}_N.$$

(15)

The mean and standard deviation of the posteriors are taken as the unfolded result in each bin in this study in order to construct the unfolded spectrum and its statistical uncertainty. Another approach is to fit the posteriors and use parameters from the fit, see Figure 3, but this method is not used in this paper, since the regularization can change the shape of the posterior to non-Gaussian.

The advantage of the FBU in comparison to other unfolding methods is in providing the whole probability distribution. Marginalizing this probability to one-dimensional posteriors the user gains the probability distribution of the unfolded result for each bin and it can decide a custom statistical property to define the unfolding result and its uncertainty in the particular bin while other methods usually provide only a single number with some uncertainty. Thus, FBU provides more user control over the unfolded output interpretation. Furthermore, marginalizing the whole probability into two dimensions enables one to study cross-bin correlations, accessible only via dedicated pseudo experiments in other methods. Another advantage is the possibility of a natural inclusion of systematic uncertainties variation by introducing prior distributions for corresponding nuisance parameters directly into the likelihood function and performing a simultaneous fit, which is a procedure known as profiling.

2.4. Motivation for the Regularization

Figure 4 illustrates the unfolding process and proceeds from the (blue) pseudo data line to the (red) unfolded line in Figure 4a, which fluctuates due to low occupied diagonal of the migration matrix M (Figure 1c). One can improve the variance of the result by introducing a regularization term aiming to diminish the curvature of the unfolding result (Figure 4b) with the optimal strength parameter ${\tau }_{\mathrm{opt}}=2089$ derived using the full sampling algorithm discussed in Section 2.3 (Figure 5), so that the resulting ${\chi }^2/\mathrm{ndf}$ between particle and unfolded spectra is minimal.

On the other hand, if one sets the strength of regularization to a very large number, e.g., $\tau \approx {10}^5$, the corresponding ${\chi }^2/\mathrm{ndf}$ is high and the unfolded result departs from the particle level spectrum, see Figure 4c.

This implies that ${\chi }^2\left(\tau \right)/\mathrm{ndf}$ may have a minimum as function of $\tau$ (see Figure 5) and the aim of this study is to provide a way to find such a minimum in a reasonable computational time. In the next section, we describe three regularization functions used in this study.

2.5. Regularization Functions $S\left(\mathbf{T}\right)$

While the regularization function can be taken as any arbitrary function, we motivate several choices [1]. The aim of the negative entropy, curvature and derivatives is to smoothen the unfolded spectra.

• Entropy regularization

  $$S_1\left(\mathbf{T}\right)=-\left[-\sum _{t=1}^N\frac{T_t}{\sum T_{t^{\prime }}}log\left(\frac{T_t}{\sum T_{t^{\prime }}}\right)\right].$$

  (16)
• Curvature regularization

  $$S_2\left(\mathbf{T}\right)=\sum _{t=2}^{N-1}{({\mathsf{\Delta }}_{t+1,t}-{\mathsf{\Delta }}_{t,t-1})}^2,$$

  (17)

  where

  $${\mathsf{\Delta }}_{t_1,t_2}=T_{t_1}-T_{t_2}.$$

  (18)
• First derivative regularization

  $$S_3\left(\mathbf{T}\right)=\sum _{t=2}^{N-1}\frac{|{\delta }_{t+1,t}-{\delta }_{t,t-1}|}{|{\delta }_{t+1,t}+{\delta }_{t,t-1}|},$$

  (19)

  where

  $${\delta }_{t_1,t_2}=\frac{\frac{T_{t_1}}{W_{t_1}}-\frac{T_{t_2}}{W_{t_2}}}{C_{t_1}-C_{t_2}}$$

  (20)

  and N is the number of bins, $W_t$ is the width of tth bin, and $C_t$ is the bin center of the tth bin.

Each of the regularization functions has a different effect on the full likelihood, see Figure 6. While sampling the likelihood according to gradient of $L_{\mathrm{reg}}\left(\mathbf{T}\right)=L\left(\mathbf{T}\right)-S\left(\mathbf{T}\right)$, i.e., leaving $\tau =1$ for the moment, the values of $S\left(\mathbf{T}\right)$ are stored.

2.6. Sampling the Likelihood Function

Effective sampling plays a crucial role in finding extremes of the likelihood function. In this paper, a private implementation of the Markov chain Hamiltonian Monte Carlo [8] method is used when each point of a function is derived from the previous point, thus forming a chain. The problem of finding the maximum of the likelihood function is transformed to generate a sample via the study of motion of a virtual particle in the potential represented by the likelihood function in an N-dimensional space.

The Hamiltonian of a particle with momentum $\mathbf{p}$ in potential $V\left(\mathbf{x}\right)$, depending on position vector $\mathbf{x}$, can be reformulated from the classical expression

$$H(\mathbf{p},\mathbf{x})=\frac{\mathbf{p}·\mathbf{p}}{2}+V\left(\mathbf{x}\right)$$

(21)

to the studied problem, where the position vector $\mathbf{x}$ is substituted by the vector of particle-level truth values $\mathbf{T}$ and the potential $V\left(\mathbf{x}\right)$ by the negative log-likelihood function $-logL\left(\mathbf{T}\right)$, so that

$$H(\mathbf{p},\mathbf{T})=\frac{\mathbf{p}·\mathbf{p}}{2}-logL\left(\mathbf{T}\right).$$

(22)

The first Hamiltonian equation of motion

$$\frac{\partial H}{\partial \mathbf{x}}=-\frac{\mathrm{d}\mathbf{p}}{\mathrm{d}t}$$

(23)

becomes in the space of truth values and momentum

$$\frac{\partial H}{\partial \mathbf{T}}=-\frac{\mathrm{d}\mathbf{p}}{\mathrm{d}t}$$

(24)

and numerically

$${\nabla }_{\mathbf{T}}logL\left(\mathbf{T}\right)=\frac{({\mathbf{p}}_{t+\epsilon /2}-{\mathbf{p}}_t)}{\epsilon /2}$$

(25)

where $\epsilon$ is the time interval. Each point in momentum space is updated as

$${\mathbf{p}}_{t+\epsilon /2}={\mathbf{p}}_t+\frac{\epsilon }{2}{\nabla }_{\mathbf{T}}log\left(L\left(\mathbf{T}\right)\right).$$

(26)

Similarly, the second equation of motion

$$\frac{\partial H}{\partial \mathbf{p}}=\frac{\mathrm{d}\mathbf{x}}{\mathrm{d}t}$$

(27)

becomes

$$\mathbf{p}=\frac{({\mathbf{T}}_{t+\epsilon }-{\mathbf{T}}_t)}{\epsilon }$$

(28)

so the truth (position) value is updated in time, according to

$${\mathbf{T}}_{t+\epsilon }={\mathbf{T}}_t+\epsilon {\mathbf{p}}_{t+\epsilon /2}$$

(29)

and the momentum as

$${\mathbf{p}}_{t+\epsilon }={\mathbf{p}}_{t+\epsilon /2}+\frac{\epsilon }{2}{\nabla }_{\mathbf{T}}logL\left({\mathbf{T}}_{t+\epsilon }\right).$$

(30)

Equations (26), (29) and (30) provide one and two-step updates for $\mathbf{T}$ and $\mathbf{p}$, respectively. The time interval $\epsilon$ is adapted in first few steps and it then stays constant for the rest of the sampling. The initial momentum $\mathbf{p}$ is chosen randomly according to normal distribution and the initial truth vector $\mathbf{T}$ is randomly chosen while using uniform distribution within limits set by user or using a default expected range in i-th bin $\left[0,3·\frac{{\left(D_{\mathrm{sim}}\right)}_i{\eta }_i}{{ϵ}_i}\right]$. These limits create a hyper-cube, in which the sampling is performed. The algorithm details are described in [8].

The introduction of the canonical momentum and shifting the N dimensional problem to $2N$ dimensions is one of the key features of the Hamiltonian MC chain generator. Classical mechanics Hamiltonian equations of motions are then employed, with the negative likelihood playing the role of a potential for a particle moving in the generalized phase-space, under the conservation of the total energy. The invariance of the system w.r.t. time translations is the motivation for exploring the trajectory of such a particle for different initial conditions, which leads to effectively sampling a larger phase space and more of the classical trajectories, thus increasing the probability of reaching the global minimum of the potential, i.e., the likelihood maximum.

3. Regularization

This section provides an overview of unfolding characteristics as function of the regularization strength parameter $\tau$. Throughout this paper, the original ${\tau }_{\mathrm{abs}}$ parameter is normalized while using the number of bins N and the curvature, entropy, or derivatives of the particle spectrum that are generally symbolized by $S\left({\mathbf{T}}_{\mathrm{part}}\right)$ as

$${\tau }_{\mathrm{abs}}=\frac{{\tau }_{\mathrm{rel}}}{(N-2)·S\left({\mathbf{T}}_{\mathrm{part}}\right)}$$

(31)

and, in the $\tau$ dependence, figures $\tau \equiv {\tau }_{\mathrm{rel}}$ are used, so different spectra and regularization functions acquire similar values of $\tau$, which are then more comparable.

3.1. Evolution of Posteriors with $\tau$

Two effects come into shaping the posteriors with stronger regularization applied. First is the shifting of the peak, which is welcome in obtaining the expected modified unfolded spectrum. The second effect decreases the posteriors width and this effect decreases the uncertainty in the mean for each bin sometimes down to an artificially infinitesimal small width with extremely high $\tau$ and, thus, undesirably somewhat increases the ${\chi }^2$ when comparing unfolded and truth spectra.

Because these are two competing effects, a minimum of the ${\chi }^2\left(\tau \right)/\mathrm{ndf}$ may or may not appear. Figure 7 shows the posterior in the 15th bin of the ${\eta }^{t\overline{t}}$ spectrum for three scenarios. First, when no regularization is applied, so $\tau =0$, then with regularization with optimal $\tau =2089$, so the final ${\chi }^2/\mathrm{ndf}$ is the smallest, and finally with an extremely high $\tau \approx {10}^4$ when the posterior becomes almost a Dirac $\delta$-function.

3.2. Evolution of Curvature, Entropy, and Derivatives with $\tau$

One of the first checks whether the regularization works properly is to study the curvature, entropy and derivatives of the unfolded spectra with respect to the increasing regularization strength $\tau$. The expected result is that, with higher $\tau$, the curvature decreases and the unfolded spectrum is smoother. The same characteristics also holds for entropy and derivatives. Figure 8 shows the decrease of curvature, entropy, and derivatives with increasing $\tau$, as expected.

3.3. Evolution of Bin Cross-Correlations with $\tau$

Using the likelihood projection to two different bins, the cross-bin correlations can be derived. Figure 9 shows the overall matrix built from these correlation factors. While, in case of (a) ($\tau =0$), the bin correlations are minimal, the progressing trend in case (b) ($\tau =912$), and finally (c) ($\tau \approx {10}^4$) is evident. Thus, the stronger regularization is the more bins of the unfolded spectrum are correlated between each other.

In order to quantify the increase of correlations, the averaged sum of off-diagonal elements of bin correlation matrix A is shown for various $\tau$, given by

$$\overline{C}\equiv \frac{1}{N_{\mathrm{off}}}\sum _{i\ne j}{A}_{ij}$$

(32)

and also its absolute value version to study the sum of correlations and anti-correlations

$${\overline{C}}_{\mathrm{abs}}\equiv \frac{1}{N_{\mathrm{off}}}\sum _{i\ne j}\left|A_{ij}\right|.$$

(33)

These bin-averaged cross correlations $\overline{C}$ and ${\overline{C}}_{\mathrm{abs}}$ are shown in Figure 10, and the increasing trend proves that, the stronger regularization, the more correlated the unfolding result over bins.

3.4. Evolution of ${\chi }^2/\mathrm{ndf}$ and Bin Uncertainties with $\tau$

The ${\chi }^2/\mathrm{ndf}$ is calculated as

$${\chi }^2/\mathrm{ndf}\equiv \frac{1}{N}\sum _i^N\frac{{(T_i-\widehat{T_i})}^2}{{\sigma }_{T_i}^2+{\sigma }_{\widehat{T_i}}^2};$$

(34)

where each ${\widehat{T}}_i$ for one particular bin is taken as the mean of the posterior and the uncertainty ${\sigma }_{{\widehat{T}}_i}$ as the standard deviation of the posterior. Because the width of the posterior is observed to decrease with rising $\tau$, the ${\chi }^2/\mathrm{ndf}$ also has to rise. In order to naively separate effects of spectra smoothing and posteriors narrowing, two variables ${\chi }_{\mathrm{num}}$ and ${\chi }_{\mathrm{denom}}$ were studied, being defined as

$${\chi }_{\mathrm{num}}\equiv \frac{1}{N}\sum _i^N{(T_i-\widehat{T_i})}^2$$

(35)

$${\chi }_{\mathrm{denom}}\equiv \frac{1}{N}\sum _i^N({\sigma }_{T_i}^2+{\sigma }_{\widehat{T_i}}^2),$$

(36)

and shown in Figure 11 and Figure 12.

If the effects of shifting and narrowing the posterior are independent then ${\chi }^2/\mathrm{ndf}$ and ratio ${\chi }_{\mathrm{num}}/{\chi }_{\mathrm{denom}}$ should be comparable

$${\chi }^2/\mathrm{ndf}\equiv \frac{1}{N}\sum _i^N\frac{{(T_i-\widehat{T_i})}^2}{{\sigma }_{T_i}^2+{\sigma }_{\widehat{T_i}}^2}\to \frac{{\chi }_{\mathrm{num}}}{{\chi }_{\mathrm{denom}}}\equiv \frac{\frac{1}{N}{\displaystyle \sum _i^N}{(T_i-\widehat{T_i})}^2}{\frac{1}{N}{\displaystyle \sum _i^N}{\sigma }_{T_i}^2+{\sigma }_{\widehat{T_i}}^2}$$

(37)

which can be seen in Figure 13.

The problem of finding the optimal $\tau$ by finding the minimal ${\chi }^2/\mathrm{ndf}$ is the time that is needed to create such a graph as in Figure 13, while each point of the graph is obtained by running the full unfolding. In the next section, an algorithm is described, which enables one to estimate the minimum of ${\chi }^2/\mathrm{ndf}$ in a much faster way.

4. Combined Sampling as a Faster Algorithm

In order to speed up the algorithm which estimates the minimum ${\chi }^2/\mathrm{ndf}$ value, the algorithm of combined sampling is proposed. The main idea is to sample separately the function $log{L}_0\left(\mathbf{T}\right)$ and the regularization function $S\left(\mathbf{T}\right)$ and then vary the $\tau$ parameter to obtain the overall likelihood

$$log{L}_{\tau }\left(\mathbf{T}\right)=log{L}_0\left(\mathbf{T}\right)-\tau S\left(\mathbf{T}\right).$$

(38)

A problem arises with sampling the regularization function $S\left(\mathbf{T}\right)$ since this function is not smooth and sampling algorithm can end up mapping only the local minima. To avoid this is, an auxiliary function

$$log{L}_1\left(\mathbf{T}\right)=log{L}_0\left(\mathbf{T}\right)-S\left(\mathbf{T}\right)$$

(39)

is sampled, where $\tau =1$ so the global minimum is dominated by $L_0\left(\mathbf{T}\right)$. The function $log{L}_1\left(\mathbf{T}\right)$ stands in sampling only for the purpose of providing a correct gradient, while values of $log{L}_0\left(\mathbf{T}\right)$ and $S\left(\mathbf{T}\right)$ are stored separately, so post-unfolding these values can be used to compute in a fast way the full desired posterior of

$$log{L}_{\mathrm{comb}}\left(\mathbf{T}\right)=log{L}_0\left(\mathbf{T}\right)-\tau S\left(\mathbf{T}\right)$$

(40)

for any $\tau$.

This faster combined sampling approach is shown in blue in Figure 14, while the red points stand for the full sampling, where, for each point in $\tau$, a new sampling had to be performed. Figure 14 shows a very good agreement between combined and full sampling methods in terms of their minima. In the case of derivative regularization in Figure 14c the full sampling method (red) does not cover the highest values of $\tau$ in the range of approximately [$5\times {10}^3$, ${10}^5$]. The reason is that the regularization function of derivatives, as compared to the curvature and entropy, is very narrow, approaching for high $\tau$ almost a $\delta$-function, see Figure 6c and, thus, the sampling either takes infinite time to find such a peak in the wide domain of the function or is stopped by reaching the maximum number of iterations and the resulting ${\chi }^2/\mathrm{ndf}$ is very high as the global maximum is not found.

5. Accidental Minima of ${\mathbf{\chi }}^{\mathbf{2}}\left(\mathbf{\tau }\right)/\mathrm{ndf}$

In the case of unfolding without regularization being effective enough so no smoothing is actually needed, the narrowing of posteriors can cause an accidental minimum in the ${\chi }^2\left(\tau \right)/\mathrm{ndf}$ function. The example of the $m^{t\overline{t}}$ spectrum shows the variable ${\chi }_{\mathrm{num}}$ as a flat function with increasing tail in high $\tau$ values, see Figure 15a. At the same time, the ${\chi }_{\mathrm{denom}}$ decreases rapidly and the ratio of these components as well as ${\chi }^2/\mathrm{ndf}$ exhibit a minimum at $\tau \approx 7000$ solely due to the narrowing of posteriors.

This accidental minimum does not improve the smoothness of the spectrum, as illustrated in Figure 16. The unfolded spectrum while using regularization in Figure 16b only decreases the uncertainties (red line) when compared to Figure 16a while not providing a smoother spectrum, as can be seen in the ratio and only a minor decrease of ${\chi }^2/\mathrm{ndf}$.

The regularization is not effective due to the fact that the tail of the $m^{t\overline{t}}$ spectrum has lower number of events compared to the first half of the spectrum. The contribution to the curvature is thus small in the tail. The solution would be to switch on the regularization only for the second half of the spectrum as shown in Figure 16c where ${\chi }^2\left(\tau \right)/\mathrm{ndf}\left(\mathrm{tail}\right)$ computed in the tail region of the spectrum decreases when compared to Figure 16a,b, also affecting the full ${\chi }^2/\mathrm{ndf}$ via a change of the global maximum of the full likelihood.

6. Hidden Minima of ${\mathbf{\chi }}^{\mathbf{2}}\left(\mathbf{\tau }\right)/\mathrm{ndf}$

If the smoothing effect that is expressed by ${\chi }_{\mathrm{num}}$ in Figure 17a has a minimum and if the narrowing effect described by ${\chi }_{\mathrm{denom}}$ in Figure 17b starts to decrease with a similar or sharper slope than ${\chi }_{\mathrm{num}}$ ($\tau \approx {10}^2$), then the real minimum in ${\chi }^2/\mathrm{ndf}$ in Figure 17c vanishes and flattens, so finding the optimal $\tau$ for the $p_T^{t,\mathrm{had}}$ spectrum in Figure 18 cannot be identified by minimizing ${\chi }^2/\mathrm{ndf}$ as a function of $\tau$. Figure 18b shows the improved unfolded spectrum that was obtained while using the minimum of ${\chi }_{\mathrm{num}}$. However, due to the narrowing of posteriors, the ${\chi }^2/\mathrm{ndf}$ increases as compared to Figure 18a (no regularization).

7. Real Minima of ${\mathbf{\chi }}^{\mathbf{2}}\left(\mathbf{\tau }\right)/\mathrm{ndf}$

If the smoothing effect expressed by ${\chi }_{\mathrm{num}}$ in Figure 19a has a global minimum and, if the narrowing effect described by ${\chi }_{\mathrm{denom}}$ in Figure 19b starts to decrease around the same region of $\tau$ values as ${\chi }_{\mathrm{num}}$ ($\tau \approx {10}^2$), then a real minimum in ${\chi }^2/\mathrm{ndf}$ in Figure 19c is observed and it helps to improve the desired property the unfolded spectrum, as shown for the spectrum of ${\eta }^{t\overline{t}}$ in Figure 20.

8. Results

The results for the spectra of $m^{t\overline{t}}$, $p_T^{t,\mathrm{had}}$ and ${\eta }^{t\overline{t}}$ were already shown in Figure 16, Figure 18 and Figure 20. In this section, remaining two studied spectra ${\eta }^{t,\mathrm{had}}$ and $p_T^{t\overline{t}}$ are shown in Figure 21 and Figure 22, where, for ${\eta }^{t,\mathrm{had}}$, the regularization is unnecessary.

The results of all five spectra and their characteristics are summarized in

Table 1. Results of minima type and basic characteristics of the migration matrix M: averaged on-diagonal factor $f_{\mathrm{diag}}$ and correlation matrix $\rho$.

Spectrum	Type	Minimum	Method	${\mathit{f}}_{\mathbf{diag}}$	$\mathit{\rho }$
$m^{t\overline{t}}$	falling	Accidental	Curvature	0.51	0.92
$p_T^{t,\mathrm{had}}$	falling	Hidden	Derivative	0.48	0.86
${\eta }^{t,\mathrm{had}}$	double-peaked	Real	Curvature	0.75	0.93
${\eta }^{t\overline{t}}$	double-peaked	Real	Curvature	0.49	0.94
$p_T^{t\overline{t}}$	falling	Real	Entropy	0.29	0.86

,

Table 2. Relative curvature, entropy and derivatives of the unfolded spectra ($\tau =0$) with respect to the curvature, entropy, and derivatives of the particle level spectra.

Spectrum	Minimum	$\frac{{\mathit{C}}_0}{{\mathit{C}}_{\mathbf{ptcl}}}$	$\frac{{\mathit{E}}_0}{{\mathit{E}}_{\mathbf{ptcl}}}$	$\frac{{\mathit{D}}_0}{{\mathit{D}}_{\mathbf{ptcl}}}$
$m^{t\overline{t}}$	Accidental	1.1	1.0	2.7
$p_T^{t,\mathrm{had}}$	Hidden	0.98	1.0	11
${\eta }^{t,\mathrm{had}}$	Real	1.1	1.0	0.96
${\eta }^{t\overline{t}}$	Real	3.7	1.0	3.5
$p_T^{t\overline{t}}$	Real	14	0.97	11

and

Table 3. Time needed to produce ${\chi }^2/\mathrm{ndf}\left(\tau \right)$ curve using combined sampling $T_{\mathrm{comb}}$ and full sampling $T_{\mathrm{full}}$ for 68 points of $\tau$ in the region [0; ${10}^5$]. Time in seconds is rounded to hundreds.

Spectrum	Minimum	${\mathit{T}}_{\mathbf{comb}}$[s]	${\mathit{T}}_{\mathbf{full}}$[s]	$\frac{{\mathit{T}}_{\mathbf{comb}}}{{\mathit{T}}_{\mathbf{full}}}$
$m^{t\overline{t}}$	Accidental	900	30,700	0.029
$p_T^{t,\mathrm{had}}$	Hidden	1200	41,300	0.030
${\eta }^{t,\mathrm{had}}$	Real	600	14,500	0.039
${\eta }^{t\overline{t}}$	Real	1000	33,700	0.029
$p_T^{t\overline{t}}$	Real	1300	44,700	0.030

in order to attempt finding a common feature that could predict the existence of real minima. Firstly, a characteristic of the migration matrix M was studied defined as

$$f_{\mathrm{diag}}\equiv \frac{1}{N}\sum _i^N{M}_{ii}=\frac{1}{N}\mathrm{Tr}M$$

(41)

where $f_{\mathrm{diag}}$ has the meaning of averaged diagonal elements. As a second characteristic, the correlation factor $\rho$ is computed while using the ROOT’s TH2D class and the GetCorrelationFactor() method [9].

Another characteristic that is shown in the Table 2 corresponds to the shape of the unfolded spectra. The curvature $C_0$, entropy $E_0$ and derivatives $D_0$ of unfolded spectra without regularization are given relative to the curvature $C_{\mathrm{ptcl}}$, entropy $E_{\mathrm{ptcl}}$, and derivatives $D_{\mathrm{ptcl}}$ of the truth particle spectra.

9. Discussion

We studied a method of finding an optimal regularization strength parameter $\tau$ and developed an independent implementation of two methods of adding regularization terms to the Fully Bayesian Unfolding and proposing a fast combined sampling method for finding an optimal value of the regularization strength parameter $\tau$. The C++ version of the Fully Bayesian Unfolding was implemented into a private modification to RooUnfold [10], which is a widely used unfolding package in HEP.

The recommended option for the user is to use the combined sampling algorithm, which returns the ${\chi }^2/\mathrm{ndf}\left(\tau \right)$ curve in a shorter time than the regular full sampling method, see Table 3.

The time that is needed to produce ${\chi }^2\left(\tau \right)$ curve while using the full sampling depends on the number of points in $\tau$ the user wishes to obtain (here, 67 points × three methods), while, using the combined method, the sampling algorithm is only run four times: first, to sample the log-likelihood function without the regularization part and afterwords three times for each of the regularization functions (curvature, entropy, and derivative), which roughly explains the time-saving ratio 4:(3×67) $\simeq 0.02$.

While no clear relation between the spectra characteristics in Table 1 and Table 2 and real minima of ${\chi }^2/\mathrm{ndf}$ was observed, the user is encouraged to evaluate ${\chi }^2/\mathrm{ndf}$, ${\chi }_{\mathrm{num}}$, and ${\chi }_{\mathrm{denom}}$ him/herself in order to convince oneself of the validity and usefulness of the regularization.

While we provided an example for the spectra of reconstructed top quarks that are produced in $pp$ collisions, the applicability of the method is not at all unique to spectra in high energy physics. Any inverse problem with finite migrations and a large number of bins is a potential candidate for benefitting from the fast sampling regularization that is introduced for the FBU method.