Optimized Probes of the CP Nature of the Top Quark Yukawa Coupling at Hadron Colliders

Abstract

We summarize our recent proposals for probing the $CP$-odd $i\tilde{\kappa }\overline{t}{\gamma }^{5}th$ interaction at the LHC and its projected upgrades directly using associated on-shell Higgs boson and top quark or top quark pair production. We first recount how to construct a $CP$-odd observable based on top quark polarization in $Wb\to th$ scattering with optimal linear sensitivity to $\tilde{\kappa }$. For the corresponding hadronic process $pp\to thj$ we then present a method of extracting the phase-space dependent weight function that allows to retain close to optimal sensitivity to $\tilde{\kappa }$. For the case of top quark pair production in association with the Higgs boson, $pp\to t\overline{t}h$, with semileptonically decaying tops, we instead show how one can construct manifestly $CP$-odd observables that rely solely on measuring the momenta of the Higgs boson and the leptons and b-jets from the decaying tops without having to distinguish the charge of the b-jets. Finally, we introduce machine learning (ML) and non-ML techniques to study the phase-space optimization of such $CP$-odd observables. We emphasize a simple optimized linear combination $\mathbf{\alpha }·\mathbf{\omega }$ that gives similar sensitivity as the studied fully fledged ML models. Using $\mathbf{\alpha }·\mathbf{\omega }$ we review sensitivity projections to $\tilde{\kappa }$ at HL-LHC, HE-LHC, and FCC-hh.

1. Introduction

The interaction between the heaviest particles of the Standard Model (SM), the top quark t and the Higgs boson h, is an important target for the LHC experiments. It is precisely predicted within the SM. The measured top quark mass $m_t$ and the electroweak condensate value v precisely determine the on-shell scalar (P- and $CP$-even) coupling $y_t=\sqrt{2}{m}_t/v$, while P- and $CP$-odd interactions are absent. Beyond the SM, effective operators of dimension-6 can break this correlation and result in more general (pseudo)scalar $\overline{t}th$ couplings $\kappa \left(\tilde{\kappa }\right)$ [1]

$${\mathcal{L}}_{ht}=-\frac{y_t}{\sqrt{2}}\overline{t}(\kappa +\mathrm{i}\tilde{\kappa }{\gamma }_5)th,$$

(1)

which reduce to the SM case at $\kappa =1$, $\tilde{\kappa }=0$.

$CP$-violating h couplings, like $\tilde{\kappa }$, are particularly interesting as any sign of $CP$ violation in Higgs processes would constitute an indisputable New Physics (NP) signal. Existing data on Higgs production and decays is already precise enough to constrain any isolated modification of the top Yukawa to $\mathcal{O}\left(1\right)$ [2,3,4]. However, all existing measurements are based on $CP$-even observables with very limited sensitivity to $CP$-odd modifications of the top quark Yukawa. In principle, indirect collider bounds from Higgs decay and production ($gg\to h$, $h\to \gamma \gamma$), and especially the low-energy bounds on electric dipole moments (EDMs) of atoms and nuclei that target specifically $CP$-odd effects [2,5,6], are currently more constraining than direct collider probes. However, these constraints are subject to assumptions about other Higgs interactions, and in particular in the case of EDMs also other contributions unrelated to the Higgs.

At the LHC it is possible to probe these couplings directly with two of the particles in Equation (1) on-shell (Since $m_h<2m_t$ one cannot probe these couplings with all the three particles on-shell) in top-Higgs associated production processes $pp\to thj$ and $pp\to t\overline{t}h$ [2,3,4,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23] (The loop induced partonic process $gg\to h\to t\overline{t}$ depends on ${\kappa }^2$, $\kappa$, and ${\tilde{\kappa }}^2$ already on the production side as it is dominated by the top quark loop [5]). The corresponding total cross sections scale as ${\kappa }^2$, ${\tilde{\kappa }}^2$ ($thj$ also as $\kappa$), and are thus poorly sensitive to small nonzero $\tilde{\kappa }$. Linear sensitivity to $\tilde{\kappa }$ on the other hand can be achieved by measuring P- and $CP$-odd observables.

We have recently addressed this challenge by identifying observables with optimal sensitivity to a single $CP$-odd parameter in both $th$ and $t\overline{t}h$ associated production at the LHC, which can be realistically measured and exhibit close to optimal sensitivity to $CP$-odd interactions between the Higgs boson and the top quark [24]. The proposed observables in $th$ are based on optimization of top-spin correlations previously studied in $t\overline{t}$ production [25]. Unfortunately, the overwhelming irreducible backgrounds make the $thj$ channel impractical. On the other hand, in the case of $\overline{t}th$ this procedure becomes intractable in practice and our construction relies instead on $CP$- and P-symmetry arguments. In total, we can identify 21 different $CP$-odd observables that can be constructed out of 5 measurable final state momenta and an additional triple-product asymmetry [26]. Namely, assuming $pp\to t\overline{t}h$ production with semileptonically decaying tops, we combine the final state lepton momenta ${\mathit{p}}_{{\ell }^{+}},{\mathit{p}}_{{\ell }^{-}}$, two b-jet momenta ${\mathit{p}}_b,{\mathit{p}}_{\overline{b}}$ from decaying top quarks (although without discriminating their charges (Efficient b-jet charge discrimination could allow to construct further $CP$-odd observables, see e.g., [6,20])) and the Higgs momentum p_h in different ways to construct C-even, P-odd laboratory frame observables w_i [24].

Due to the high dimensionality and complexity of the phase-space in this process with top quarks decaying semileptonically, using the complete kinematical information accessible experimentally to construct an optimal $CP$-odd observable is challenging. To this end we have employed neural networks (NN) trained on Monte-Carlo generated samples to efficiently parametrize the weight function of events across the multi-dimensional phase-space in order to maximize the statistical sensitivity to $\tilde{\kappa }$ [27]. We show how the required P- and $CP$-symmetry properties of the NN-based observables can be imposed a priori. Finally, we compare in terms of optimality, a general $CP$-odd NN function of the phase-space to a linear combination of manifestly $CP$-odd variables.

The present paper serves as a pedagogical review of the work first reported in Refs. [24,27].

2. Optimal CP-Odd Observable in the bW → th Process

2.1. Parton Level $Wb\to th$ Analysis

We begin by studying the effects of $\tilde{\kappa }$ on top spin observables in the idealized case of single top quark production in partonic process $W\left(p\right)b\left(p^{\prime }\right)\to t\left(k\right)h\left(k^{\prime }\right)$. Here the complete polarized scattering amplitudes can be found in a compact analytic form. This process can actually be connected to a more realistic $pp\to thj$ production in the high energy limit, where the W and b quark mass effects are negligible and the collinear emission of both initial state ‘partons’ can be described by the corresponding parton distribution functions. (See e.g., Section 3 of Ref. [28] for an extended discussion on the validity of this approximation) Three diagrams contribute to such parton level Higgs-top production in the SM, shown in Figure 1. Neglecting furthermore the mass (and thus the corresponding Yukawa coupling) of the bottom quark, we consider only the first two of the diagrams in Figure 1. The formalism presented here is based on Refs. [25,29,30]. First, we introduce the spin projection operator [31]

$$P\left(s\right)=\frac{1}{2}(1+{\gamma }_5\cancel{s}),$$

(2)

where $s^{\mu }$ is a top spin four-vector, defined in a general frame as

$$s^{\mu }=\left(\frac{\mathit{k}·\widehat{\mathit{s}}}{m_t},\widehat{\mathit{s}}+\frac{\mathit{k}(\mathit{k}·\widehat{\mathit{s}})}{m_t(E_t+m_t)}\right).$$

(3)

Vector $\mathit{k}$ is the top quark momentum and $\widehat{\mathit{s}}$ is an arbitrary unit vector that represents polarization of the top quark. The physical significance of $\widehat{\mathit{s}}$ is revealed if we make a rotation-free boost to the top rest frame where we find $s^{*\mu }=(0,\widehat{\mathit{s}})$. Note that spatial component of four-vector $x^{\mu }$ in this case transforms as ${\mathit{x}}^{*}=\mathit{x}+\left(\frac{\mathit{x}·\mathit{k}}{m_t(E_t+m_t)}-\frac{x^0}{m_t}\right)\mathit{k}$ upon boost to the top rest frame. Therefore $s^2=-1$, $s·k=0$, and $\widehat{\mathit{s}}$ corresponds to the polarization of the top quark in its rest frame. Projection onto a well defined polarization of the top quark is achieved by placing the projector (2) in front of the bispinors at the amplitude level:

$$\begin{array}{cc}\hfill u& \to P\left(s\right)u,\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill v& \to P\left(s\right)v.\hfill \end{array}$$

(5)

Sum over top quark polarizations r then becomes

$$\begin{array}{cc}\hfill \sum _{r}u(k,r)\overline{u}(k,r)& \to P\left(s\right)(\cancel{k}+m)P\left(s\right)=(\cancel{k}+m)P\left(s\right),\hfill \\ \hfill \sum _{r}v(k,r)\overline{v}(k,r)& \to P\left(s\right)(\cancel{k}-m)P\left(s\right)=(\cancel{k}-m)P\left(s\right),\hfill \end{array}$$

(6)

and the cross-section is linear in $s^{\mu }$

$${\left|\mathcal{M}\right|}^2=a+b_{\mu }{s}^{\mu }.$$

(7)

Here the polarization vector $\widehat{\mathit{s}}$ is arbitrary while $b_{\mu }$ contains all the information about the polarization of the top quark in the given process. The parton level cross section can be written as

$$d\sigma ={\mathsf{\Phi }}_{\mathrm{in}}{\left|\mathcal{M}\right|}^{2}d{\mathsf{\Gamma }}_{th},$$

(8)

where ${\mathsf{\Phi }}_{\mathrm{in}}$ is the initial state flux normalization and $d{\mathsf{\Gamma }}_{th}$ is the $th$ phase space volume. On the other hand, in the top rest frame it is convenient to introduce the spin density matrix as

$$\rho =A+B_i{\sigma }_i,$$

(9)

such that the unpolarized cross section is proportional to $\overline{{\left|\mathcal{M}\right|}^2}=\mathrm{Tr}\left[\rho \right]=2A$. Here $\mathbf{\sigma }$ are the Pauli matrices. In the density matrix formalism, the expectation value of a generic operator is obtained as

$$〈\mathcal{O}〉=\mathrm{Tr}\left[\rho \mathcal{O}\right].$$

(10)

In particular, the polarized cross section along $\widehat{\mathit{s}}$ is obtained as the expectation value of the projector:

$${\left|\mathcal{M}\right|}^2=\mathrm{Tr}\left[\rho \frac{1}{2}(1+\widehat{\mathit{s}}·\mathbf{\sigma })\right]=A+B_i{\widehat{s}}_i.$$

(11)

One can determine the rest-frame coefficients $A,B_i$ from a, $b_{\mu }$ by comparing the expressions for polarized ${\left|\mathcal{M}\right|}^2$, expressed via Equations (7) and (11). The result of this matching are explicit expressions:

$$A=a,B_i=-b^i+\frac{1}{m_t}\left(b^0-\frac{\mathit{b}·\mathit{k}}{E_t+m_t}\right)\mathit{k}.$$

(12)

The rest-frame polarization of the top quark along a vector $\widehat{\mathit{s}}$ is given by the expectation value of ${\mathcal{O}}_{\widehat{\mathit{s}}}=\widehat{\mathit{s}}·\frac{\mathbf{\sigma }}{2}$,

$$〈{\mathcal{O}}_{\widehat{\mathit{s}}}〉=\mathit{b}·\widehat{\mathit{s}}.$$

(13)

This observable can be determined for example by measuring the angular distribution of the charged lepton in the semi-leptonic top decay $t\to b(W\to \ell \nu )$ since the charged lepton in top decay is considered to be an almost perfect top spin analyzer, i.e., the angular decay distribution vanishes when the lepton momentum is opposite to the spin of t [32]. The angular distribution in the top rest frame

$$\frac{1}{{\mathsf{\Gamma }}_t}\frac{d{\mathsf{\Gamma }}_t}{dcos{\theta }_{\ell }}=\frac{1}{2}\left(1+〈{\mathcal{O}}_{\widehat{\mathit{s}}}〉cos{\theta }_{\ell }\right)$$

(14)

then allows for experimental extraction of the $B^i$ coefficients [32].

Here ${\theta }_{\ell }$ is an angle between the lepton and the polarization axis $\widehat{\mathit{s}}$ in the top rest frame. The above construction shows that the vector $\widehat{\mathit{s}}$ is an arbitrary unit vector defined in the laboratory frame. A particular choice $\widehat{\mathit{s}}=\mathit{k}/\left|\mathit{k}\right|$ implies that $〈{\mathcal{O}}_{\widehat{\mathit{s}}}〉$ measures the top quark helicity. Another natural choice for $\widehat{\mathit{s}}$ is the W momentum $\widehat{\mathit{p}}$, also known as the beam basis, which has to be carefully defined in actual $pp$ collisions where the (symmetric) initial state does not allow to define forward and backward directions. Experimentally one has to reconstruct the top quark rest frame in order to be able to trace the angular distribution of the lepton with respect to the chosen $\widehat{\mathit{s}}$ and gain access to the coefficients $B_i$. In the following we will show how to optimize the choice of $\widehat{\mathit{s}}$ such that the sensitivity to the $CP$-violating parameter $\tilde{\kappa }$ is maximized.

In the $Wb$ center-of-mass frame we can define the W and t momenta as

$$\begin{array}{cc}\hfill \widehat{\mathit{p}}& =(0,0,1),\hfill \\ \hfill \widehat{\mathit{k}}& =(sin\theta ,0,cos\theta ),\hfill \end{array}$$

(15)

where $\theta$ is the angle between the direction of the top quark and the W boson. We have set the azimuthal angle $\varphi =0$ without loss of generality. The polarization vector components $B_i$ in this case depend on $x=cos\theta$. Evaluation of the two diagrams in Figure 1 leads to the coefficients $B_{1,2,3}\left(x\right)$ of the polarized cross-section. It turns out that in the coordinate system in Equation (15) the analytical expression for $B_2\left(x\right)$ is linear in $\tilde{\kappa }$,

$$B_2\left(x\right)\equiv \beta \left(x\right)\tilde{\kappa },$$

(16)

whereas $B_{1,3}$ do not contain linear $\tilde{\kappa }$ terms. Effectively this means that we should choose the vector $\widehat{\mathit{s}}$ to be orthogonal to the plane spanned by the W and t momenta in order to probe $\tilde{\kappa }$ with linear sensitivity. Similar results have been reported in Ref. [18]. In pursuit of maximal sensitivity to $\tilde{\kappa }$ we choose the polarization vector in each event as $\widehat{\mathit{s}}=\widehat{\mathit{p}}\times \widehat{\mathit{k}}/|\widehat{\mathit{p}}\times \widehat{\mathit{k}}|$. In this case an interesting experimental quantity a two-fold differential cross-section

$$\begin{array}{cc}\hfill \frac{d^2\sigma }{dxdcos{\theta }_{\ell }}(Wb\to hb\ell \nu )& =\Sigma (x,\widehat{\mathit{s}})\frac{\mathrm{Br}(t\to b\ell \nu )}{2}(1+cos{\theta }_{\ell })\hfill \\ \hfill & \phantom{=}+\Sigma (x,-\widehat{\mathit{s}})\frac{\mathrm{Br}(t\to b\ell \nu )}{2}(1-cos{\theta }_{\ell }),\hfill \end{array}$$

(17)

where we have approximated the intermediate top quark as a narrow resonance and $\Sigma (x,\widehat{\mathit{s}})=d\sigma /dx(Wb\to t^{\left(\widehat{\mathit{s}}\right)}h)$ is the differential production cross section for the top quarks polarized in the $\widehat{\mathit{s}}$ direction. Using Equation (11) and inserting $\widehat{\mathit{s}}$ we have $\Sigma (x,\pm \widehat{\mathit{s}})={\mathsf{\Phi }}_{\mathrm{in}}(A\left(x\right)\pm \tilde{\kappa }\beta \left(x\right))$, where ${\mathsf{\Phi }}_{\mathrm{in}}$ is the initial flux normalization. Thus we can write Equation (17) as

$$\frac{d^2\sigma }{dxdcos{\theta }_{\ell }}(Wb\to hb\ell \nu )={\mathsf{\Phi }}_{\mathrm{in}}\mathrm{Br}(t\to b\ell \nu )(A\left(x\right)+\tilde{\kappa }\beta \left(x\right)cos{\theta }_{\ell }).$$

(18)

Treating $\tilde{\kappa }$ as a small perturbation we can integrate the distribution in Equation (18) with a phase-space dependent function f that would maximize statistical sensitivity of the integral to $\tilde{\kappa }$. It has been shown in Refs. [33,34] that such an optimal function should be the ratio of the $\tilde{\kappa }$-perturbation to the unperturbed distribution, in our case $f(x,cos{\theta }_{\ell })=\frac{\beta \left(x\right)}{A\left(x\right)}cos{\theta }_{\ell }$. The optimal observable is thus

$${\mathcal{O}}_{\mathrm{opt}}^{Wb\to th}.\equiv \frac{1}{\sigma }\int dxdcos{\theta }_{\ell }\frac{d^2\sigma }{dxdcos{\theta }_{\ell }}\frac{\beta \left(x\right)}{A\left(x\right)}cos{\theta }_{\ell }=\frac{1}{N}\sum _{i=1}^N\frac{\beta \left(x_i\right)}{A\left(x_i\right)}cos{\theta }_{\ell ,i},$$

(19)

where ${\theta }_{\ell }$ is the angle between $\widehat{\mathit{s}}$ and the lepton momentum in the top center-of mass-frame, as defined in the preceding paragraph. The index $i=1,\dots ,N$ labels individual events. The prediction scales as $〈{\beta }^2〉$,

$${\mathcal{O}}_{\mathrm{opt}.}^{Wb\to th}=\frac{\tilde{\kappa }}{3}\left[\int dx\frac{{\left[\beta \left(x\right)\right]}^2}{A\left(x\right)}\right]/\left[\int dxA\left(x\right)\right],$$

(20)

where we have integrated over $cos{\theta }_{\ell }$ and left the bounds for $x=cos\theta$ unspecified. The function $\beta \left(x\right)$ is plotted in Figure 2.

To carry over the presented formalism to the realistic case of $pp$ collisions, we have to adapt the beam axis by referring only to experimentally accessible momenta. Using the reconstructed top momentum $\mathit{k}$ as a reference, we define the positive z-direction as the parallel top quark momentum projection ${\widehat{\mathit{k}}}_{\Vert }$. The top quark is then by definition in the positive hemisphere, $\tilde{x}=cos\tilde{\theta }\ge 0$, where $\tilde{\theta }$ is the angle between $\mathit{k}$ and ${\widehat{\mathit{k}}}_{\Vert }$. The optimal polarization direction with linear $\tilde{\kappa }$ sensitivity now becomes $\widehat{\mathit{s}}={\widehat{\mathit{k}}}_{\Vert }\times {\widehat{\mathit{k}}}_{\perp }$ upon which an experiment should measure the lepton angle ${\tilde{\theta }}_{\ell }$. The cross-section distributions in $\tilde{x}$ and x are related via

$$\begin{array}{cc}\hfill \frac{d^2\sigma }{d\tilde{x}dcos{\tilde{\theta }}_{\ell }}& ={\left.\frac{d^2\sigma }{dxdcos{\theta }_{\ell }}\right|}_{x=\tilde{x},cos{\theta }_{\ell }=cos{\tilde{\theta }}_{\ell }}+{\left.\frac{d^2\sigma }{dxdcos{\theta }_{\ell }}\right|}_{x=-\tilde{x},cos{\theta }_{\ell }=-cos{\tilde{\theta }}_{\ell }}\hfill \\ \hfill & ={\mathsf{\Phi }}_{\mathrm{in}}\frac{\mathrm{Br}(t\to b\ell \nu )}{2}\left[\tilde{A}\left(\tilde{x}\right)+\tilde{\kappa }cos{\tilde{\theta }}_{\ell }\tilde{\beta }\left(\tilde{x}\right)\right],\hfill \end{array}$$

(21)

where

$$\begin{array}{cc}\hfill \tilde{A}\left(\tilde{x}\right)& \equiv A\left(\tilde{x}\right)+A(-\tilde{x}),\hfill \\ \hfill \tilde{\beta }\left(\tilde{x}\right)& \equiv \beta \left(\tilde{x}\right)-\beta (-\tilde{x}).\hfill \end{array}$$

(22)

The $cos{\theta }_{\ell }$ is flipped in the second term since for $\tilde{x}=-x$ the polarization vector $\widehat{\mathit{s}}={\widehat{\mathit{k}}}_{\Vert }\times {\widehat{\mathit{k}}}_{\perp }$ flips the direction compared to the previous definition, $\widehat{\mathit{s}}\sim \mathit{p}\times \mathit{k}$. The optimal observable in this case is finally

$$\begin{array}{cc}\hfill {\tilde{\mathcal{O}}}_{\mathrm{opt}.}^{Wb\to th}& \equiv \frac{1}{\sigma }\int d\tilde{x}dcos{\tilde{\theta }}_{\ell }\frac{d^2\sigma }{d\tilde{x}dcos{\tilde{\theta }}_{\ell }}cos{\tilde{\theta }}_{\ell }\frac{\tilde{\beta }\left(\tilde{x}\right)}{\tilde{A}\left(\tilde{x}\right)}\hfill \\ \hfill & =\frac{\tilde{\kappa }}{3}\left[\int d\tilde{x}\frac{{\left[\tilde{\beta }\left(\tilde{x}\right)\right]}^2}{\tilde{A}\left(\tilde{x}\right)}\right]/\left[\int d\tilde{x}\tilde{A}\left(\tilde{x}\right)\right],\hfill \end{array}$$

(23)

where the optimal weight function is again taken to be the ratio of the $\tilde{\kappa }$-perturbation to the unperturbed distribution (see Equation (21)), in line with Refs. [33,34]. In terms of experimental N data points with reconstructed ${\tilde{\theta }}_{\ell }$ and $\tilde{x}$ the observable is obtained as the following average:

$${\tilde{\mathcal{O}}}_{\mathrm{opt}.}^{Wb\to th}=\frac{1}{N}\sum _{i=1}^N\frac{\tilde{\beta }\left({\tilde{x}}_i\right)}{\tilde{A}\left({\tilde{x}}_i\right)}cos{\tilde{\theta }}_{\ell ,i}.$$

(24)

In the limit where $\beta \left(x\right)=-\beta (-x)$ the observables are equal, ${\tilde{\mathcal{O}}}_{\mathrm{opt}.}^{Wb\to th}={\mathcal{O}}_{\mathrm{opt}.}^{Wb\to th}$. However in general the ${\tilde{\mathcal{O}}}_{\mathrm{opt}.}^{Wb\to th}$ is expected to result in a weaker statistical significance due to our inability to determine the direction of the top quark with respect to the initial W. In Figure 2 we show that $\beta \left(x\right)$ is large at negative x and we have $\tilde{\beta }\left(\tilde{x}\right)\approx -\beta (-\tilde{x})$, for representative values of the center-of-mass energy $\sqrt{s}$.

2.2. Hadronic Process $pp\to thj$

Here we demonstrate how to carry over the optimal $CP$-violating observable from parton level to the realistic case of $pp$ collisions, but still neglecting reconstruction efficiencies and backgrounds. The parton level observable defined in Equation (23) can be adapted to this case with an additional integration over the parton distribution functions (PDFs). Since the hadronic cross section is a convolution of partonic cross sections it can be split into a $\tilde{\kappa }$-independent piece and the small perturbation proportional to $\tilde{\kappa }$, similar to the partonic cross section in Equation (21). Assuming that the Higgs decays into visible states, the missing $p_T$ is only due to the neutrino originating from the top decay. Thus we can reconstruct the top quark momentum and kinematic quantities of Equation (21). For hadronic collisions one can express the cross section as

$$\frac{d^2{\sigma }^{pp\to thj}}{d\tilde{x}dcos{\tilde{\theta }}_{\ell }}=\mathcal{A}\left(\tilde{x}\right)+\tilde{\kappa }cos{\tilde{\theta }}_{\ell }\mathcal{B}\left(\tilde{x}\right),$$

(25)

and weigh the events with the optimal $f_{\mathrm{opt}}.\propto cos{\tilde{\theta }}_{\ell }\mathcal{B}/\mathcal{A}$. To demonstrate the procedure we have used the Monte Carlo event generator $\mathtt{MadGraph}\mathtt{5}$ [35,36] together with the Higgs Characterisation UFO model [37,38] (for an analysis of next-to-leading order QCD and next-to-next-to leading EW effects see Refs. [8,17,39], respectively) to incorporate the $\kappa$ and $\tilde{\kappa }$ couplings in the simulation of the $pp\to t(\to b\ell \nu )hj$ signal. The procedure of extracting the optimal weight function $\mathcal{B}/\mathcal{A}$ from MC simulations and using it to produce the optimal observable goes as follows:

• Choose the bins for $\tilde{x}$ between ${\tilde{x}}_{\min }\ge 0$ and ${\tilde{x}}_{\max }\le 1$.
• Fix $\tilde{\kappa }$ and extract from the MC simulation the mean $\langle cos{\tilde{\theta }}_{\ell }\rangle$ in each of the $\tilde{x}$ bins. The obtained value corresponds to the weight $\frac{1}{3}\mathcal{B}/\mathcal{A}$ in this bin, see Equation (25).
• Use this information to weigh experimental events bin-by-bin with $f_{\mathrm{opt}}.\propto cos{\tilde{\theta }}_{\ell }\mathcal{B}/\mathcal{A}$. The normalization of $f_{\mathrm{opt}}.$ is fixed by the requirement $\int d\tilde{x}\mathcal{B}/\mathcal{A}=1$.

This optimization procedure is independent of the $\tilde{\kappa }$ value as long as $\tilde{\kappa }$ is sufficiently small. The resulting optimal weight $\mathcal{B}/\mathcal{A}$ is shown in Figure 3, where we compare it for different final states ($thj$ or $\overline{t}hj$) and collision energies (14 or $27\mathrm{TeV}$). To assess the stability of the proposed method against higher order corrections we have extracted the weight function from simulations at NLO in QCD to estimate the systematic uncertainty associated with QCD effects and found that the difference is within 10% of the LO extraction. Finally, we can compare our optimized approach to the naïve $\tilde{\kappa }$ extraction through the measurement of ${\mathcal{O}}_{\mathrm{na}ï\mathrm{ve}}=〈cos{\theta }_{\ell }〉$, which in turn corresponds to the case where the weight is independent of $\tilde{x}$, i.e., $\mathcal{B}/\mathcal{A}=1$. We define the signal significance of an observable $\mathcal{O}$ as

$$\mathrm{Sig}\left(\tilde{\kappa }\right)=\frac{\mathcal{O}\left(\tilde{\kappa }\right)}{{\sigma }_{\mathcal{O}}\left(\tilde{\kappa }\right)}.$$

(26)

In Figure 4 we show the improvement of the significance when the optimal weight function is applied on simulated signal events without showering or reconstruction effects at 14 TeV.

2.3. Limits in the $(\kappa ,\tilde{\kappa })$ Plane from $pp\to thj$ at Event Reconstruction Level

In order to make closer contact with experiments, we can also include the effects of parton showering, detector response and background processes. In our analysis we have used $\mathtt{MadGraph}\mathtt{5}$ to generate events at leading order (LO) in QCD for the signal process $pp\to t(\to b\ell \nu )h(\to b\overline{b})j$ plus the conjugate process with $\overline{t}$ at 14 TeV High-Luminosity LHC (HL-LHC) and 27 TeV High-Energy LHC (HE-LHC) center-of-mass energies. (Note that our procedure of obtaining an optimal observable does not depend on the h decay products, therefore this analysis should be taken as a proof of concept with potential for future improvements using e.g., multiple h decay channels) Event generation was performed for multiple values of ($\kappa$, $\tilde{\kappa })$. The parton level events were subsequently showered and hadronized with $\mathtt{Pythia}\mathtt{8}$ [40], and jets are clustered with the anti-$k_T$ algorithm using $\mathtt{FastJet}$ [41]. For detector simulation and final state object reconstruction (e.g., lepton isolation and b-tagging) we used $\mathtt{Delphesv}\mathtt{3}.\mathtt{3}.\mathtt{3}$ [42] with the default ATLAS parameters in $\mathtt{delphes}\_\mathtt{card}\_\mathtt{ATLAS}.\mathtt{tcl}$. The dominant background process in this analysis is $t\overline{t}$ production with additional associated jets. We included this background by generating $pp\to t\overline{t}$ samples, with one of the tops decayed into the semi-leptonic channel and the other one decayed into the hadronic channel, produced in association with 0, 1, and 2 hard jets. In order to correctly model the hard jets’ distributions, we merged the matrix element computations with the MC shower using the $\mathtt{MLM}$ [43] prescription. For the event selection we demand the following basic requirements:

• Exactly 3 b-tagged jets with $\left|\eta \right(b\left)\right|<5$ and $p_T\left(b\right)>20$ GeV,
• One additional (non-tagged) light jet exclusively in the forward direction with $2<\left|\eta \right(j\left)\right|<5$ and $p_T\left(j\right)>20$ GeV,
• One isolated light lepton ${\ell }^{\pm }=e^{\pm },{\mu }^{\pm }$ with $\left|\eta \right(\ell \left)\right|<2.5$ and $p_T(\ell )>10$ GeV.

In addition, we further select events with one reconstructed Higgs and one reconstructed top quark as follows: first, we calculate the three possible invariant masses from the three reconstructed b-jets ($m_{bb}$) and only keep the event if at least one $bb$ pair satisfies $|m_H-m_{bb}|<15$ GeV. For such events, we select as the Higgs decay candidate $h\to b\overline{b}$ for the pair of b-jets with the invariant mass closest to the Higgs mass. The remaining non-Higgs b-jet is then assumed to come from the top-quark decay. Next, we reconstruct the top-quark by requiring that the combined invariant mass $m_{bl\nu }$ of the remaining b-jet, the lepton, and the neutrino (also reconstructed by assuming it to be the unique source of missing energy in the event) to fall inside the mass window of the top-quark defined by $m_t\pm 35$ GeV. In order to further reject the $t\overline{t}$ backgrounds, events with a reconstructed Higgs and top are selected if the combined invariant mass of the b-jets originating from the Higgs and the light jet satisfies the cut $m_{bbj}>280$ GeV [44]. The final selection efficiency for the $thj$ signal in the SM is $0.32\%$ ($0.23\%$), while for the background it is $0.008\%$ ($0.006\%$) at 14 TeV (27 TeV).

As we fully reconstruct the $th$ system and have access to the lepton momentum from the top decay we have all the necessary information for measuring the optimized spin observable. We use the optimal weight function $\mathcal{B}/\mathcal{A}$ (Figure 3) extracted from the MC simulations to construct a ${\chi }^2$ with an appropriately weighted signal process. Our results for $pp\to thj$ generated in the SM are given by the $2\sigma$ exclusion limits (shaded blue) shown in Figure 5 for the HE-LHC at a luminosity of $15{\mathrm{ab}}^{-1}$. As can be seen in Equation (23) the observable ${\tilde{\mathcal{O}}}_{\mathrm{opt}}.$ is normalized to the cross section, which contains terms ${\kappa }^2$, ${\tilde{\kappa }}^2$, as well as a linear term in $\kappa$ and a constant term due to second diagram in Figure 1, whereas the numerator is proportional to $\tilde{\kappa }(\kappa +c)$. The behaviour of ${\mathcal{O}}_{\mathrm{opt}}$ close to the SM point is thus linear in $\tilde{\kappa }$, whereas the cross section has a minimum in $\kappa$ close to $\kappa =1$. In the large coupling regime ${\tilde{\mathcal{O}}}_{\mathrm{opt}}.$ converges to a small value which depends on the direction in which we make the limit ${\kappa }^2+{\tilde{\kappa }}^2\to \infty$. The $2\sigma$ exclusion has an elliptic shape as shown by the blue contour in Figure 5. We also present the limit (given by the black elliptic contour) assuming a $2\sigma$ positive excess above the SM expectation corresponding to a measurement of the optimized spin observable of ${\tilde{\mathcal{O}}}_{\mathrm{opt}.}=0.06\pm 0.03$ whose size and error are statistics-driven. Because of the nature of our observable, the signed fluctuation gives rise to asymmetric limits in the $\tilde{\kappa }$ direction. In the $\kappa$ direction the bounds are also not symmetric as $pp\to thj$ production contains linear $\kappa$ terms. Finally, in order to include background effects, the same statistical analysis would have to be repeated including the $t\overline{t}$ background in the ${\chi }^2$ fit. However, even with a large background rejection as implemented above, the irreducible background is simply too large and the signal is completely diluted leading to a signal significance of only $S/\sqrt{B}\sim 0.8\left(3.2\right)$ at 14 TeV (27 TeV) at a luminosity of $3{\mathrm{ab}}^{-1}$ ($15{\mathrm{ab}}^{-1}$). This effectively precludes any meaningful extraction of bounds on $\tilde{\kappa }$ from a fit to ${\tilde{\mathcal{O}}}_{\mathrm{opt}.}$. We leave the possibility of further optimizing the cuts in order to reduce the backgrounds or including other Higgs decay channels as a future challenge. In the remainder of this review we instead focus on the related but more abundant process of associated top quark pair and Higgs boson production.

3. CP-Odd Observables in the $\mathbf{pp}\to \mathbf{t}\overline{\mathbf{t}}\mathbf{h}$ Process

In this Section we introduce laboratory-frame accessible and phase-space optimized $CP$-odd observables in the process $pp\to t\overline{t}h$ in which both top quarks decays semi-leptonically: $t\to b{l}^{+}\nu$ and $\overline{t}\to \overline{b}{l}^{-}\overline{\nu }$. This process has a higher S/B ratio compared to $pp\to thj$ and has indeed been measured by the LHC collaborations [45,46] (For the state of the art predictions of the differential distributions see e.g., Ref. [39]) The top quarks in $pp\to t\overline{t}h$ are known to be unpolarized, independent of the value of $\tilde{\kappa }$ [2]. The information on the underlying $\kappa$ and $\tilde{\kappa }$ parameters is in principle contained in the correlations among the top quark spins, however due to the experimental difficulties of extracting the top quark polarizations in this process, this approach is unfeasible. This is the reason we focus in this Section on manifestly $CP$-odd observables, directly sensitive to $\tilde{\kappa }$ and easily accessible by measurements of final-state momenta in the lab frame.

3.1. Laboratory Frame $CP$-Odd Observables in $t\overline{t}h$ Production

The accessible final-state momenta [6] of this decay in the lab frame are 3-momenta of b-jet ${\mathit{p}}_b$, $\overline{b}$-jet ${\mathit{p}}_{\overline{b}}$, 3-momenta of leptons ${\mathit{p}}_{{\ell }^{+}}$ and ${\mathit{p}}_{{\ell }^{-}}$, and 3-momenta of the Higgs ${\mathit{p}}_h$. In practice differentiating between b-jet and b-jet is difficult (For recent attempts in extracting the charge of theb-jet see References [47,48,49,50]), therefore we should consider only observables which are invariant under p_b ↔ p_b transformation. We construct these observables from 3-vector quantities with well defined C and P eigenvalues, that are given in

Table 1. Vector quantities with well-defined C and P eigenvalues in 3 dimensional Euclidean space. More complicated objects with well-defined C and P eigenvalues can be constructed using variables in this table.

	ph	${\mathit{p}}_{{\mathit{\ell }}^{-}}+{\mathit{p}}_{{\mathit{\ell }}^{+}}$	${\mathit{p}}_{{\mathit{\ell }}^{-}}-{\mathit{p}}_{{\mathit{\ell }}^{+}}$	${\mathit{p}}_{\mathit{b}}+{\mathit{p}}_{\overline{\mathit{b}}}$	${\mathit{p}}_{\mathit{b}}-{\mathit{p}}_{\overline{\mathit{b}}}$
C	+	+	−	+	−
P	−	−	−	−	−
$CP$	−	−	+	−	+

.

We use quantities in Table 1 to construct $CP$-odd variables $\omega$ of mass-dimension 5, that are C-even and invariant under ${\mathit{p}}_b\leftrightarrow {\mathit{p}}_{\overline{b}}$ transformation. In order to systematically obtain all distinct $\omega$’s we proceed as follows. First, we construct variables of the form $\omega \sim V_1\times V_2·V_3{V}_4·V_5$ (Notice that the possibility of a nested cross product $(((V_1\times V_2)\times V_3)\times V_4)·V_5$ can also be reduced to this form) using $V_j\in \{{\mathit{p}}_h,{\mathit{p}}_{{\ell }^{-}}+{\mathit{p}}_{{\ell }^{+}},{\mathit{p}}_{{\ell }^{-}}-{\mathit{p}}_{{\ell }^{+}},{\mathit{p}}_b+{\mathit{p}}_{\overline{b}},{\mathit{p}}_b-{\mathit{p}}_{\overline{b}}\}$ for $j\in \{1,...,5\}$. Doing so, we find 150 potential quintuple products. We symmetrize them with respect to C-conjugation and ${\mathit{p}}_b\leftrightarrow {\mathit{p}}_{\overline{b}}$ transformation. The non-zero quintuple products are $\omega$ variables, however they may be linearly dependent. Indeed some of the obtained $\omega$’s are connected via the following Euclidean identity

$${\delta }_{ab}{ϵ}_{cde}-{\delta }_{ac}{ϵ}_{deb}+{\delta }_{ad}{ϵ}_{ebc}-{\delta }_{ae}{ϵ}_{bcd}=0,$$

(27)

which can be written as:

$$\mathit{a}(\mathit{b}\times \mathit{c}·\mathit{d})-\mathit{b}(\mathit{c}\times \mathit{d}·\mathit{a})+\mathit{c}(\mathit{d}\times \mathit{a}·\mathit{b})-\mathit{d}(\mathit{a}\times \mathit{b}·\mathit{c})=0,$$

(28)

where $\mathit{a}$, $\mathit{b}$, $\mathit{c}$ and $\mathit{d}$ are four arbitrary vectors in 3 dimensional Euclidean space. The sign of individual terms in the last expression corresponds to the sign of the cyclic permutation of the four vectors.

The first class of $\omega$’s involves ${\mathit{p}}_{{\ell }^{+}}$ and ${\mathit{p}}_{{\ell }^{-}}$ in the mixed product, ${\mathit{p}}_{{\ell }^{-}}-{\mathit{p}}_{{\ell }^{+}}$ in the scalar product. Both products are invariant under ${\mathit{p}}_b\leftrightarrow {\mathit{p}}_{\overline{b}}$:

$$\begin{array}{cc}\hfill {\omega }_1& \sim \left[({\mathit{p}}_{{\ell }^{-}}\times {\mathit{p}}_{{\ell }^{+}})·{\mathit{p}}_h\right]\left[({\mathit{p}}_{{\ell }^{-}}-{\mathit{p}}_{{\ell }^{+}})·{\mathit{p}}_h\right],\hfill \end{array}$$

(29)

$$\begin{array}{cc}\hfill {\omega }_2& \sim \left[({\mathit{p}}_{{\ell }^{-}}\times {\mathit{p}}_{{\ell }^{+}})·{\mathit{p}}_h\right]\left[({\mathit{p}}_{{\ell }^{-}}-{\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_{{\ell }^{-}}+{\mathit{p}}_{{\ell }^{+}})\right],\hfill \end{array}$$

(30)

$$\begin{array}{cc}\hfill {\omega }_3& \sim \left[({\mathit{p}}_{{\ell }^{-}}\times {\mathit{p}}_{{\ell }^{+}})·{\mathit{p}}_h\right]\left[({\mathit{p}}_{{\ell }^{-}}-{\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_b+{\mathit{p}}_{\overline{b}})\right],\hfill \end{array}$$

(31)

$$\begin{array}{cc}\hfill {\omega }_4& \sim \left[({\mathit{p}}_{{\ell }^{-}}\times {\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_b+{\mathit{p}}_{\overline{b}})\right]\left[({\mathit{p}}_{{\ell }^{-}}-{\mathit{p}}_{{\ell }^{+}})·{\mathit{p}}_h\right],\hfill \end{array}$$

(32)

$$\begin{array}{cc}\hfill {\omega }_5& \sim \left[({\mathit{p}}_{{\ell }^{-}}\times {\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_b+{\mathit{p}}_{\overline{b}})\right]\left[({\mathit{p}}_{{\ell }^{-}}-{\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_{{\ell }^{-}}+{\mathit{p}}_{{\ell }^{+}})\right],\hfill \end{array}$$

(33)

$$\begin{array}{cc}\hfill {\omega }_6& \sim \left[({\mathit{p}}_{{\ell }^{-}}\times {\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_b+{\mathit{p}}_{\overline{b}})\right]\left[({\mathit{p}}_{{\ell }^{-}}-{\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_b+{\mathit{p}}_{\overline{b}})\right].\hfill \end{array}$$

(34)

The second class involves ${\mathit{p}}_b\times {\mathit{p}}_{\overline{b}}$ and/or ${\mathit{p}}_b-{\mathit{p}}_{\overline{b}}$ in both mixed and scalar products:

$$\begin{array}{cc}\hfill {\omega }_7& \sim \left[({\mathit{p}}_b\times {\mathit{p}}_{\overline{b}})·{\mathit{p}}_h\right]\left[({\mathit{p}}_b-{\mathit{p}}_{\overline{b}})·{\mathit{p}}_h\right],\hfill \end{array}$$

(35)

$$\begin{array}{cc}\hfill {\omega }_8& \sim \left[({\mathit{p}}_b\times {\mathit{p}}_{\overline{b}})·{\mathit{p}}_h\right]\left[({\mathit{p}}_b-{\mathit{p}}_{\overline{b}})·({\mathit{p}}_{{\ell }^{-}}+{\mathit{p}}_{{\ell }^{+}})\right],\hfill \end{array}$$

(36)

$$\begin{array}{cc}\hfill {\omega }_9& \sim \left[({\mathit{p}}_b\times {\mathit{p}}_{\overline{b}})·{\mathit{p}}_h\right]\left[({\mathit{p}}_b-{\mathit{p}}_{\overline{b}})·({\mathit{p}}_b+{\mathit{p}}_{\overline{b}})\right],\hfill \end{array}$$

(37)

$$\begin{array}{cc}\hfill {\omega }_{10}& \sim \left[({\mathit{p}}_b\times {\mathit{p}}_{\overline{b}})·({\mathit{p}}_{{\ell }^{-}}+{\mathit{p}}_{{\ell }^{+}})\right]\left[({\mathit{p}}_b-{\mathit{p}}_{\overline{b}})·{\mathit{p}}_h\right],\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill {\omega }_{11}& \sim \left[({\mathit{p}}_b\times {\mathit{p}}_{\overline{b}})·({\mathit{p}}_{{\ell }^{-}}+{\mathit{p}}_{{\ell }^{+}})\right]\left[({\mathit{p}}_b-{\mathit{p}}_{\overline{b}})·({\mathit{p}}_{{\ell }^{-}}+{\mathit{p}}_{{\ell }^{+}})\right],\hfill \end{array}$$

(39)

$$\begin{array}{cc}\hfill {\omega }_{12}& \sim \left[({\mathit{p}}_b\times {\mathit{p}}_{\overline{b}})·({\mathit{p}}_{{\ell }^{-}}+{\mathit{p}}_{{\ell }^{+}})\right]\left[({\mathit{p}}_b-{\mathit{p}}_{\overline{b}})·({\mathit{p}}_b+{\mathit{p}}_{\overline{b}})\right],\hfill \end{array}$$

(40)

$$\begin{array}{cc}\hfill {\omega }_{13}& \sim \left[({\mathit{p}}_b\times {\mathit{p}}_{\overline{b}})·({\mathit{p}}_{{\ell }^{-}}-{\mathit{p}}_{{\ell }^{+}})\right]\left[({\mathit{p}}_b-{\mathit{p}}_{\overline{b}})·({\mathit{p}}_{{\ell }^{-}}-{\mathit{p}}_{{\ell }^{+}})\right],\hfill \end{array}$$

(41)

$$\begin{array}{cc}\hfill {\omega }_{14}& \sim \left[({\mathit{p}}_{{\ell }^{-}}\times {\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_b-{\mathit{p}}_{\overline{b}})\right]\left[({\mathit{p}}_b-{\mathit{p}}_{\overline{b}})·({\mathit{p}}_{{\ell }^{-}}-{\mathit{p}}_{{\ell }^{+}})\right].\hfill \end{array}$$

(42)

The third class involves mixed product of ${\mathit{p}}_h,{\mathit{p}}_{{\ell }^{-}}\pm {\mathit{p}}_{{\ell }^{+}}$, and ${\mathit{p}}_b\pm {\mathit{p}}_{\overline{b}}:$:

$$\begin{array}{cc}\hfill {\omega }_{15}& \sim \left[{\mathit{p}}_h\times ({\mathit{p}}_{{\ell }^{-}}+{\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_b-{\mathit{p}}_{\overline{b}})\right]\left[({\mathit{p}}_b-{\mathit{p}}_{\overline{b}})·{\mathit{p}}_h\right],\hfill \end{array}$$

(43)

$$\begin{array}{cc}\hfill {\omega }_{16}& \sim \left[{\mathit{p}}_h\times ({\mathit{p}}_{{\ell }^{-}}+{\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_b-{\mathit{p}}_{\overline{b}})\right]\left[({\mathit{p}}_b-{\mathit{p}}_{\overline{b}})·({\mathit{p}}_{{\ell }^{-}}+{\mathit{p}}_{{\ell }^{+}})\right],\hfill \end{array}$$

(44)

$$\begin{array}{cc}\hfill {\omega }_{17}& \sim \left[{\mathit{p}}_h\times ({\mathit{p}}_{{\ell }^{-}}+{\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_b-{\mathit{p}}_{\overline{b}})\right]\left[({\mathit{p}}_b-{\mathit{p}}_{\overline{b}})·({\mathit{p}}_b+{\mathit{p}}_{\overline{b}})\right],\hfill \end{array}$$

(45)

$$\begin{array}{cc}\hfill {\omega }_{18}& \sim \left[{\mathit{p}}_h\times ({\mathit{p}}_{{\ell }^{-}}-{\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_b+{\mathit{p}}_{\overline{b}})\right]\left[({\mathit{p}}_{{\ell }^{-}}-{\mathit{p}}_{{\ell }^{+}})·{\mathit{p}}_h\right],\hfill \end{array}$$

(46)

$$\begin{array}{cc}\hfill {\omega }_{19}& \sim \left[{\mathit{p}}_h\times ({\mathit{p}}_{{\ell }^{-}}-{\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_b+{\mathit{p}}_{\overline{b}})\right]\left[({\mathit{p}}_{{\ell }^{-}}-{\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_{{\ell }^{-}}+{\mathit{p}}_{{\ell }^{+}})\right],\hfill \end{array}$$

(47)

$$\begin{array}{cc}\hfill {\omega }_{20}& \sim \left[{\mathit{p}}_h\times ({\mathit{p}}_{{\ell }^{-}}-{\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_b+{\mathit{p}}_{\overline{b}})\right]\left[({\mathit{p}}_{{\ell }^{-}}-{\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_b+{\mathit{p}}_{\overline{b}})\right],\hfill \end{array}$$

(48)

$$\begin{array}{cc}\hfill {\omega }_{21}& \sim \left[{\mathit{p}}_h\times ({\mathit{p}}_{{\ell }^{-}}-{\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_b-{\mathit{p}}_{\overline{b}})\right]\left[({\mathit{p}}_{{\ell }^{-}}-{\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_b-{\mathit{p}}_{\overline{b}})\right].\hfill \end{array}$$

(49)

There are further possibilities with a mixed product ${\mathit{p}}_h\times ({\mathit{p}}_{{\ell }^{-}}+{\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_b+{\mathit{p}}_{\overline{b}})$ multiplied by one of 8 C-even, $b\leftrightarrow \overline{b}$ even scalar products $\{{\mathit{p}}_h·{\mathit{p}}_h,{\mathit{p}}_h·({\mathit{p}}_{{\ell }^{-}}+{\mathit{p}}_{{\ell }^{+}}),{\mathit{p}}_h·({\mathit{p}}_b+{\mathit{p}}_{\overline{b}}),({\mathit{p}}_{{\ell }^{-}}+{\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_b+{\mathit{p}}_{\overline{b}}),{({\mathit{p}}_{{\ell }^{-}}\pm {\mathit{p}}_{{\ell }^{+}})}^2,{({\mathit{p}}_b\pm {\mathit{p}}_{\overline{b}})}^2\}$. Since the mixed product itself already has the desired symmetry properties, those 8 quintuple products do not bring additional new information, with respect to a mixed (triple) product that is our final observable:

$${\omega }_{22}\sim {\mathit{p}}_h\times ({\mathit{p}}_{{\ell }^{-}}+{\mathit{p}}_{{\ell }^{+}})·({\mathit{p}}_b+{\mathit{p}}_{\overline{b}}).$$

(50)

Note that there are nonlinear relations between $\omega$’s, such as ${\omega }_1{\omega }_6={\omega }_3{\omega }_4$, that we do not exploit to further reduce the set. Namely, ratios of $\omega$’s can contain singularities in the available phase-space and as such would be difficult to reconstruct by a neural network optimizer.

All the $\omega$’s are normalized by the lengths of the vectors that enter as factors in the scalar products,

$${\omega }_i=\frac{[{\mathit{V}}_1\times {\mathit{V}}_2·{\mathit{V}}_3][{\mathit{V}}_4·{\mathit{V}}_5]}{|{\mathit{V}}_1\times {\mathit{V}}_2\left|\right|{\mathit{V}}_3\left|\right|{\mathit{V}}_4\left|\right|{\mathit{V}}_5|},$$

(51)

and the upper bound $|{\omega }_i|\le 1$ is generally valid. For cases when ${\omega }_i$ has a vector $\mathit{a}$ present both in the mixed and scalar products, e.g., $({\mathit{V}}_1\times {\mathit{V}}_2·\mathit{a})({\mathit{V}}_3·\mathit{a})$, and furthermore with ${\mathit{V}}_1\times {\mathit{V}}_2·{\mathit{V}}_3=0$, a stricter upper bound $|{\omega }_i|\le 1/2$ applies (for ${\omega }_{1,6,7,11,13,14,15,16,20,21}$).

Having constructed manifestly $CP$-odd variables $\omega$, we now show how they can be used to extract information on $\tilde{\kappa }$ in an optimal way. The $t\overline{t}h$ production cross section can be written as

$$\frac{d\sigma }{d\mathit{x}d{\omega }_i}=A(\mathit{x},|{\omega }_i\left|\right)+\tilde{\kappa }\kappa B(\mathit{x},{\omega }_i).$$

(52)

where $\mathit{x}$ are $CP$-even phase space variables, A is a manifestly $CP$-even and B is a manifestly $CP$-odd function of ${\omega }_i$: $B(\mathit{x},{\omega }_i)=-B(\mathit{x},-{\omega }_i)$. The $\kappa \tilde{\kappa }$ dependence is due to the interference of scalar and pseudoscalar amplitudes. A simple $CP$-odd observable is an average of a single variable ${\omega }_i$

$${\mathcal{O}}_i=\frac{1}{\sigma }\int d\mathit{x}d{\omega }_i\frac{d\sigma }{d\mathit{x}d{\omega }_i}{\omega }_i=\frac{1}{N}\sum _{j=1}{\omega }_i^{\left(j\right)},$$

(53)

where N is the number of experimental events. The standard deviation ${\sigma }_i$ of such an observable is given by

$${\sigma }_i^2=\frac{1}{N}\left[\frac{1}{N}\sum _j{\left({\omega }_i^{\left(j\right)}\right)}^2-{\mathcal{O}}_i^2\right].$$

(54)

For large enough N the distribution of ${\mathcal{O}}_i$ is approximately Gaussian and the corresponding significance of this observable is:

$${\mathrm{Sig}}_i\equiv \frac{{\mathcal{O}}_i}{{\sigma }_i}=\sqrt{N}\frac{{\mathcal{O}}_i}{\sqrt{\frac{1}{N}{\sum }_j{\left({\omega }_i^{\left(j\right)}\right)}^2-{\mathcal{O}}_i^2}}.$$

(55)

Equation (55) holds for ${\omega }_1,...,{\omega }_{22}$ and their linear combinations. By studying the behavior of all 22 such observables ${\mathcal{O}}_i$ on MC simulations of $pp\to t\overline{t}h$ with semi-leptonically decaying top quarks we have found that ${\mathcal{O}}_6$ and ${\mathcal{O}}_{14}$ are the most promising in terms of their significance. Our next aim is to show how one can achieve better sensitivity to $\tilde{\kappa }$ by introducing optimized observables in two directions: the phase-space optimization of a single ${\omega }_i$ and a construction of an optimal $CP$-odd observable as a combination of all available $\omega$s.

3.2. NN-Based Optimized $CP$-Odd Observables

Due to the high dimensionality and complexity of the phase-space in the $t\overline{t}h$ process with top quarks decaying semileptonically, we rely on neural networks (NN) trained on Monte-Carlo generated samples to efficiently parametrize the weight function of events across the multi-dimensional phase-space in order to maximize the statistical sensitivity to $\tilde{\kappa }$. We note that existing general purpose ML inference tools are already able to optimize sensitivity to a given parameter (see e.g., Ref. [51]). The purpose of our work was to do so in an economic way that manifestly respects the symmetries of the problem.

We implemented the training and evaluation of neural networks using the $\mathtt{TensorFlow}$ framework [52]. In all cases, we used a sample of ${10}^7$$pp\to ht(\to b{\ell }^{+}\nu )\overline{t}(\to \overline{b}{\ell }^{-}\overline{\nu })$ events generated at LO using $\mathtt{Madgraph}\mathtt{5}$ [35] together with the Higgs Characterisation UFO model [37,38] with $\kappa ,\tilde{\kappa }=1$. We split the sample into separate training (7.5 M) and test (2.5 M) samples. After training at fixed $\tilde{\kappa }=1$ we also tested the observables at other values of $\tilde{\kappa }$ and $\kappa$, both ranging from $-1$ to 1. In these tests 1 M events have been used.Unless stated otherwise the results are shown for events in $pp$ collisions at 14 TeV. We randomly initialized the neural network weights using the default Glorot uniform initializer and used the Adam optimizer with a custom varying learning rate $l\left(e\right)=l(e-1)/(1+0.8^e)$ where e is the current epoch and the initial learning rate is set to $0.1$. We use $\mathtt{relu}$ for the activation function. We trained all networks using the novel loss function

$$\mathrm{loss}\left(\mathbf{\alpha }\right)={\left(\frac{\mathrm{mean}\left(\mathcal{F}\right(\mathit{X};\mathbf{\alpha }\left)\right)}{\mathrm{std}\left(\mathcal{F}(\mathit{X};\mathbf{\alpha })\right)/\sqrt{N}}\right)}^{-2},$$

(56)

where the $\mathrm{mean}\left(\right)$ and the standard deviation $\mathrm{std}\left(\right)$ are to be calculated over all events in the sample. The loss corresponds to the inverse of the significance-squared of the observable $\mathrm{mean}\left(\mathcal{F}\right(\mathit{X};\mathbf{\alpha }\left)\right)$ that should be minimized in order to achieve optimal statistical sensitivity. Here N is the size of the event sample, $\mathbf{\alpha }$ are the free neural network weights and biases and $\mathit{X}$ stands for the values of $CP$-even and/or $CP$-odd phase-space variables in the given event. We emphasize that such a choice of the loss function is unique to the problem at hand - we are using the optimization procedure to directly maximize the significance of each considered observable. We can avoid over-fitting of the training sample by stopping the training when at least 30 epochs have passed and one of the following two criteria is satisfied: either the running average of 20 training losses saturates to $0.5\%$ or the running average of 20 test losses increases for 5 epochs in a row. We keep a model history and in the end choose the best model in terms of test loss. In practice we found that mostly the first condition terminates the training loop, and the best model is usually the model from the final epoch of training. In order to determine the optimal NN architecture we performed a scan over a set of possible NN configurations with up to 2 hidden layers and up to 9 nodes per NN layer. (We have also considered an automated algorithm to determine the optimal NN architecture (i.e., Hyperopt [53], see also Ref. [54] for one of its recent uses.). Here instead we present results of manual scans over a set of possible NN configurations in order to have better control over the NN parameters. We found the results of both approaches comparable. We choose this cutoff for representational purposes, however we have checked that our results do not change significantly when using larger networks, namely up to three hidden layers of 30 nodes each.)

3.2.1. Phase-Space Optimization of a Single $\omega$

First we present the optimization of the ${\omega }_6$ and ${\omega }_{14}$ variables based on phase-space averaging. We do not follow the optimization procedure based on separating A and B in Equation (52) since this would require cumbersome multidimensional binning [33]. We use a vector of easily accessible $CP$-even Mandelstam variables $\mathit{x}$:

$$\mathit{x}=\left(\begin{array}{c}({\mathit{p}}_{{\ell }^{+}}+{\mathit{p}}_{{\ell }^{-}})·{\mathit{p}}_h\\ ({\mathit{p}}_{{\ell }^{+}}+{\mathit{p}}_{{\ell }^{-}})·({\mathit{p}}_b+{\mathit{p}}_{\overline{b}})\\ ({\mathit{p}}_b+{\mathit{p}}_{\overline{b}})·{\mathit{p}}_h\\ {\mathit{p}}_{{\ell }^{+}}·{\mathit{p}}_{{\ell }^{-}}\\ {\mathit{p}}_b·{\mathit{p}}_{\overline{b}}\end{array}\right).$$

(57)

Our goal is to find the optimal $CP$-even weight function $f(\mathit{x};\mathbf{\alpha })$, which should be used to calculate the weighted average of ${\omega }_i$. The function f takes $CP$-even quantities $\mathit{x}$ as inputs, therefore we expect its dependence on $\tilde{\kappa }$ to be of the form

$$f(\mathit{x};\mathbf{\alpha })=C(\mathit{x};\mathbf{\alpha })+{\tilde{\kappa }}^{2}D(\mathit{x};\mathbf{\alpha })+\mathcal{O}\left({\tilde{\kappa }}^4\right).$$

(58)

Using (52) we can now express the observable

$$\begin{array}{cc}\hfill \langle f(\mathit{x};\mathbf{\alpha }){\omega }_i\rangle & =\frac{1}{\sigma }\int \frac{d\sigma }{d\mathit{x}d{\omega }_i}f(\mathit{x};\mathbf{\alpha }){\omega }_{i}d\mathit{x}d{\omega }_i\hfill \\ \hfill & =\frac{\tilde{\kappa }\kappa }{\sigma }\int B(\mathit{x},{\omega }_i)C(\mathit{x};\mathbf{\alpha }){\omega }_{i}d\mathit{x}d{\omega }_i+\frac{{\tilde{\kappa }}^3\kappa }{\sigma }\int B(\mathit{x},{\omega }_i)D(\mathit{x};\mathbf{\alpha }){\omega }_{i}d\mathit{x}d{\omega }_i+\mathcal{O}\left({\tilde{\kappa }}^5\right),\hfill \end{array}$$

(59)

with the usual definition of the average (The phase space average of a function is defined as $\langle \#\rangle \equiv \frac{1}{\sigma }\int \frac{d\sigma }{d\mathit{x}d\omega }\#d\mathit{x}d\omega$). The integration region in $\mathit{x}-{\omega }_i$ space is symmetric with respect to the transformation ${\omega }_i\to -{\omega }_i$, therefore integrals of the arguments which are anti-symmetric in ${\omega }_i$ vanish. Due to this reason all contributions to the expected value of the observable $\langle f(\mathit{x};\mathbf{\alpha }){\omega }_i\rangle$ are proportional to odd powers of $\tilde{\kappa }$. The large dimensionality of the phase-space suggests the parameterisation of the function $f(\mathit{x};\mathbf{\alpha })$ by means of an appropriate NN. In terms of the loss function (56) we have $\mathcal{F}(\mathit{x},{\omega }_i;\mathbf{\alpha })=f(\mathit{x};\mathbf{\alpha }){\omega }_i$.

To understand the impact of using different possible neural network architectures, we have performed a manual scan over a set of neural network configurations. The input layer has 5 nodes (one per each $\mathit{x}$ component) and the output layer has one node resulting in a scalar $f(\mathit{x};\mathbf{\alpha })$. We studied networks with a single hidden layer of 1–9 nodes and double hidden layer networks with 1–9 nodes each, constraining the number of nodes on the second hidden layer to be smaller than or at most equal to the number of nodes on the first hidden layer. The results of the converged test losses of 50 different random weight initializations per configuration are shown on Figure 6 in the purple box plot for the case of ${\omega }_6$ and the orange box plot for the case of ${\omega }_{14}$. The plain ${\omega }_6$-based observable is shown in gray, with the dashed lines denoting its $1\sigma$ statistical uncertainty, while the same holds true for plain ${\omega }_{14}$ in black. We found that the phase-space optimization of ${\omega }_6$ gives a noticeable improvement over plain ${\omega }_6$ when using a large enough network. Interestingly, ${\omega }_{14}$ seems to be close to optimal on its own, as the phase space optimization does not introduce noticeable improvement. Moreover, the optimized ${\omega }_6$ gives a similar performance to ${\omega }_{14}$, hinting that we have reached maximal performance achievable with a single $\omega$.

To test how well the resulting networks generalize to other values of $\tilde{\kappa }$ we used the 50 converged $\{9,9\}$ models to calculate the dependence of the resulting observable significances with respect to $\tilde{\kappa }$ on the aforementioned fresh sets of 1 M events per $\tilde{\kappa }$. This is shown on Figure 7 where a consistent improvement over simple $\langle {\omega }_6\rangle$ can be seen at all considered $\tilde{\kappa }$, while again a marginal improvement is confirmed for ${\omega }_{14}$, with the optimized ${\omega }_6$ hovering around ${\omega }_{14}$.

As the results of optimizing single ${\omega }_6$ and ${\omega }_{14}$ point to a maximal performance possible using a single $\omega$ and the chosen set of phase space variables, we now turn to the rest of the $\omega$’s. In the next subsection we consider a more general case where the $CP$-odd observable itself is parameterized with a neural network.

3.2.2. Neural Network as a $CP$-Odd Observable

We consider the case where the output of the neural network is a $CP$-odd quantity that defines our observable. To this end, we build a network with 22 inputs, one per each ${\omega }_i$, and one output $F(\mathbf{\omega };\mathbf{\alpha })$, which is correctly anti-symmetrized so that $F(\mathbf{\omega };\mathbf{\alpha })=-F(-\mathbf{\omega };\mathbf{\alpha })$. In terms of the loss function (56) we have $\mathcal{F}(\mathit{x};\mathbf{\alpha })=F(\mathbf{\omega };\mathbf{\alpha })$. Note that since we include the complete irreducible set $\mathbf{\omega }$ in this non-linear construction, it effectively also covers the case of a simple phase-space optimization of any (linear combination of) ${\omega }_i$, since all relevant $CP$-even phase-space variables can be recovered by taking suitable products of ${\omega }_i{\omega }_{i^{\prime }}$.

We again carried out the study of the dependence of the network size with respect to the test sample loss, including non-negligible uncertainties associated with random weight initializations. We scanned the neural network architecture parameter space in the same way as in the previous case, starting with a single hidden layer of 1–9 nodes, then adding an additional hidden layer with the number of nodes smaller than or equal to the number of nodes on the first hidden layer. For each configuration we ran 50 trainings with different random weight initializations. The results are shown in Figure 6 in the blue box plot. We find a considerable improvement over the phase-space optimizations of ${\omega }_6$ or ${\omega }_{14}$. The improvement is consistent in the entire range of $\tilde{\kappa }\in [0.1,1.0]$ and is most striking at large $\tilde{\kappa }$.

Again we can check the generalizing power of the resulting observables to other $\tilde{\kappa }$ by fixing the model configuration to $\{9,9\}$ and calculating the significance of the resulting observables with respect to $\tilde{\kappa }$. The results are shown on Figure 7. We find a consistent improvement over the previous case across all considered $\tilde{\kappa }$. A noticeable improvement in the significance can be seen. In order to better understand the physics underlying the optimization, we next consider this model in the leading order approximation in $\mathbf{\omega }$.

3.2.3. First Order Approximation of $F($$\mathbf{\omega }$;$\mathbf{\alpha }$)

To address the arbitrariness of the neural network architecture choice and to better understand the underlying physics, we finally consider the first order approximation of the function $F(\mathbf{\omega };\mathbf{\alpha })$, which can be expanded in a Taylor series $F(\mathbf{\omega };\mathbf{\alpha })={\sum }_j{\alpha }_j{\omega }_j+\mathcal{O}\left({\omega }^3\right)$ for $j\in \{1,\dots ,22\}$, since for most events the values of the $CP$-odd variables are small: $|{\omega }_i|\ll 1$, $i\in \{1,...,22\}$. In the first order approximation the optimal $CP$-odd observable can be written as

$${\mathcal{O}}_{\alpha ·\omega }=\langle \sum _j{\alpha }_j{\omega }_j\rangle ,$$

(60)

where ${\alpha }_j$ are parameters that satisfy a subsidiary condition $\left|\mathbf{\alpha }\right|\equiv {\sum }_{j=1}^{22}{\alpha }_j^2=1$. Their values can be found by maximizing the significance:

$$\frac{\partial }{\partial {\alpha }_k}\frac{{\mathcal{O}}_{\alpha ·\omega }}{\mathrm{std}(\alpha ·\omega )}=0,$$

(61)

The left side of the expression above can be expanded in the following way:

$$\begin{array}{cc}\hfill \frac{\partial }{\partial {\alpha }_k}\frac{{\mathcal{O}}_{\alpha ·\omega }}{\mathrm{std}(\alpha ·\omega )}& =\frac{2\langle \mathbf{\alpha }·\mathbf{\omega }\rangle }{\mathrm{std}(\alpha ·\omega )}\left\{\langle {\omega }_k\rangle \langle {(\mathbf{\alpha }·\mathbf{\omega })}^2\rangle -\langle (\mathbf{\alpha }·\mathbf{\omega }){\omega }_k\rangle \right\}\hfill \\ \hfill & =\frac{2\langle \mathbf{\alpha }·\mathbf{\omega }\rangle }{\mathrm{std}(\alpha ·\omega )}\left\{\sum _{i,j=1}^{22}{\alpha }_i{\alpha }_j\left[\langle {\omega }_i{\omega }_k\rangle \langle {\omega }_j\rangle -\langle {\omega }_i{\omega }_j\rangle \langle {\omega }_k\rangle \right]\right\}.\hfill \end{array}$$

(62)

where in the last step we take $\mathbf{\alpha }$ outside the $\langle -\rangle$, for example $\langle \mathbf{\alpha }·\mathbf{\omega }\rangle ={\sum }_j{\alpha }_j\langle {\omega }_j\rangle$. By assuming $\tilde{\kappa }\ne 0$ we have ${\mathcal{O}}_{\mathbf{\alpha }·\mathbf{\omega }}\ne 0$. From this, together with the condition in Equation (61), it follows that the expression in the curly brackets of Equation (62) must be equal to 0. Hence, we obtain a system of 22 quadratic equations (The problem is equivalent to a single neuron NN with 22 inputs and one output without the activation function and the bias term)

$${\alpha }^T{M}^{\left(k\right)}\alpha =0,$$

(63)

where $\alpha ={[{\alpha }_1,...,{\alpha }_{22}]}^T$ and the $22\times 22$ matrices $M^{\left(k\right)}$ are given by (Notice that by definition $M_{ik}^{\left(k\right)}=0$ for all i and k, therefore $det\left(M^{\left(k\right)}\right)=0$ for all k)

$$M_{ij}^{\left(k\right)}=\langle {\omega }_i{\omega }_k\rangle \langle {\omega }_j\rangle -\langle {\omega }_i{\omega }_j\rangle \langle {\omega }_k\rangle .$$

(64)

The system of equations in Equation (63) can be solved numerically. The solution vector $\mathbf{\alpha }$ is undetermined up to a real non-zero constant. We normalize it such that $\mathbf{\alpha }·\mathbf{\alpha }=1$ and choose the solution with mostly positive values.

We used this approach to extract the optimal weights ${\alpha }_j$ from ${10}^7$ events generated with $\tilde{\kappa }=\kappa =1$ at 14, 27, and 100 TeV. We also estimated the uncertainty associated with the optimal weights using the following procedure: First we estimate the statistical spread of the significance obtained with optimal $\mathbf{\alpha }$. Next we allow a single ${\alpha }_j$ to float in the intervals $[{\alpha }_j-{\sigma }_j,{\alpha }_j+{\sigma }_j]$, where ${\sigma }_j$ is chosen such that the decrease of the significance due to the change in ${\alpha }_j$ corresponds to the statistical spread of the significance. We perform an efficient scan around the optimal vector $\mathbf{\alpha }$ in its 22-dimensional neighborhood using spherical coordinates to trivially fulfill the normalization constraint ${\sum }_j{\alpha }_j^2=1$. We approximate the significance with a quadratic function around the extremum to find independent, uncorrelated directions in the $\alpha$-space. With this procedure we determine how sharply the optimal ${\alpha }_j$ are defined. In practice, we estimated the statistical error of the significance using ${10}^7$ events. Clearly the uncertainties ${\sigma }_j$ are larger for smaller chosen sample size. The results of this approach are shown in Figure 8, where the upper (lower) panel shows the estimated error (significance) for each ${\alpha }_j$ at 14, 27, and 100 TeV. A comparison of the observable $\mathbf{\alpha }·\mathbf{\omega }$ to other approaches discussed previously is shown in Figure 7. We reach a similar level of improvement compared to the full $F(\mathbf{\omega };\mathbf{\alpha })$ network with significantly fewer parameters.

3.3. Bounds in the $(\kappa ,\tilde{\kappa })$ Plane

We could now produce the bounds in the $(\kappa ,\tilde{\kappa })$ plane by varying both $\kappa$ and $\tilde{\kappa }$ at generator level and including showering and hadronization effects using $\mathtt{Pythia}\mathtt{8}$ and detector effects using $\mathtt{Delphes}$ with the default ATLAS simulation card. As the $t\overline{t}h$ is followed by semileptonic top decays and $h\to b\overline{b}$ decay, our signal is defined as 4 b-jets and two oppositely charged leptons ℓ. We included the main irreducible background $pp\to t\overline{t}b\overline{b}$ with both tops decaying semileptonically and used the event selection requirements:

• $\ge 4$ jets of any flavor with $\left|\eta \right(j\left)\right|<5$ and $p_T\left(j\right)>20$ GeV.
• $\ge 3$ of the above jets are b-tagged.
• 2 oppositely charged light leptons with $\left|\eta \right(\ell \left)\right|<2.5$ and $p_T(\ell )>10$ GeV.

Furthermore, in order to identify the b-jets from t, $\overline{t}$ decays we counted the number $N_b$ of tagged b-jets and performed the following selections: if $N_b\ge 4$, we compute the invariant masses $m_{bb}$ of all possible b-jet pairs and select the pair with invariant mass closest to the Higgs mass $m_h=125$ GeV. If the selected pair falls inside the Higgs mass window ($|m_{bb}-m_h|<15$ GeV) we removed the pair from the list of b-jets and selected from this list the highest $p_T$b-jets as our candidate top quark decay b-jets. However, if $N_b=3$ we computed all possible invariant masses $m_{bj}$ where j are non-b jets in the event. We selected as the $h\to b\overline{b}$ candidate the $bj$ pair that minimizes $|m_h-m_{bj}|$ and falls inside the Higgs mass window $m_h\pm 15$ GeV. The remaining two b-jets are taken as the candidate top quark decay b-jets.

We present the bounds for HL- and HE-LHC, FCC-hh by using the optimal observable (60) with the weights shown in the upper plot of Figure 8. The bounds coming from a null result up to the expected statistical uncertainty for different luminosities at different energies are shown on Figure 9, where we also show the expected sensitivity to $(\kappa ,\tilde{\kappa })$ from the $t\overline{t}h$ production cross-section measurements using the projected uncertainties of $\delta \kappa /\kappa \sim (0.04,0.02,0.009)$ at HL-LHC, HE-LHC [55] and FCC-hh [56], respectively. For direct comparison with [24] we also show the expected bounds from using a single ${\omega }_6$ and have checked that using ${\omega }_{14}$ does not change the single $\omega$ bounds significantly.

A consistent improvement of sensitivity compared to using single ${\omega }_i$ can be achieved by using the optimized combination of $\omega$’s, constraining the parameter space significantly in orthogonal directions compared to the cross-section measurements. Interestingly the significance improvement is consistent between partonic events and after including shower, detector effects, as well as the dominant background and realistic object reconstruction, even though the optimization was performed at parton level only. This robustness is a welcome benefit of the method, since the computationally costly optimization procedure does not appear to be sensitive to modeling of the hadronic final states and detector effects. It is also reassuring that the optimization does not significantly rely on specific phase-space regions particularly affected by the background. We expect the results to be also robust against higher order QCD corrections as in the case of $pp\to thj$, where we have checked this explicitly.

We show the sensitivity of the optimized observable to the sign of $\tilde{\kappa }$ (and $\kappa$) in Figure 10 by assuming the measurement of a $2\sigma$ positive statistical fluctuation of the SM case, which in our estimate corresponds to the measurement of $\mathbf{\alpha }·\mathbf{\omega }=(4.6\pm 2.3)\times {10}^{-4}$, $\mathbf{\alpha }·\mathbf{\omega }=(0.9\pm 0.45)\times {10}^{-4}$ and $\mathbf{\alpha }·\mathbf{\omega }=(0.2\pm 0.1)\times {10}^{-4}$ for HL-LHC (3 ab${}^{-1}$), HE-LHC (15 ab${}^{-1}$) and FCC-hh (30 ab${}^{-1}$) respectively.

4. Conclusions

In order to establish, directly and with minimal additional assumptions, the presence of a $CP$-odd component of the top quark Yukawa ($\tilde{\kappa }$), we have studied manifestly $CP$-odd observables in $th$ and $t\overline{t}h$ production at the LHC and its prospective upgrades.

For the $thj$ final states we have relied on the possibility of reconstructing the t quark momentum and accessing the t polarization. We have identified a particular polarization direction which is perpendicular to the $th$ plane, where the top polarization along this direction would undoubtedly point to the presence of the $CP$-odd coupling $\tilde{\kappa }$. We have presented a method for optimizing the phase space dependent weight and shown its sensitivity at the HL- and HE-LHC for the semileptonic top and $h\to b\overline{b}$ mode. The handful of signal events offer discriminating power, sensitive to the sign of $\tilde{\kappa }$, however the irreducible background due to $t\overline{t}$+jets severely dilutes the sensitivity of the proposed observable.

On the other hand, $t\overline{t}h$ production has a considerably larger cross section at LHC energies compared to $thj$, while suffering more moderately from irreducible backgrounds. Due to the complexity of the final state kinematics with multiple undetected particles we have in this case proposed variables ${\omega }_i$ that only depend on the lab-frame accessible momenta and are manifestly P- and $CP$-odd. Furthermore, we have studied the prospect of their phase-space optimization, parameterizing the optimal weight functions with neural networks. In particular, we have studied a general $CP$-odd observable, parameterized directly by a $CP$ anti-symmetric neural network, which results in better performance compared to any individual ${\omega }_i$. Finally, we have studied the first order approximation of this network as a linear combination of the $CP$-odd observables, producing a simpler and more robust observable, $\mathbf{\alpha }·\mathbf{\omega }$. One benefit of using $\mathbf{\alpha }·\mathbf{\omega }$ optimized at parton level is that it retains close to optimal sensitivity to the $CP$-odd coupling even after detector simulation, event selection and reconstruction, allowing to probe $\tilde{\kappa }$ directly at HL-LHC, HE-LHC and FCC-hh. We have found that, at the end of Run 3, the LHC will exclude $\kappa \tilde{\kappa }\sim 1$ with $2\sigma$ confidence, while FCC-hh will be sensitive to $\kappa \tilde{\kappa }\sim 0.01$. Note that these observables represent highly complementary probes of the top Yukawa sector compared to $pp\to t\overline{t}h$ cross-section measurements. In particular in their optimized form they would allow to break the degeneracy in the $(\kappa ,\tilde{\kappa })$ plane and significantly reduce the allowed parameter space even at modest sensitivities accessible at the (HL)LHC.