

 21 May 1996

CEBAF-TH-96-04 TECHNION-PH-96-11 BNL CP Nonconservation in p _p ! t_bX at the Tevatron David Atwooda, Shaouly Bar-Shalomb, Gad Eilamb and Amarjit Sonic

a) Theory Group, CEBAF, Newport News, VA 23606, USA

b) Department of Physics, Technion, Haifa, Israel c) Department of Physics, Brookhaven National Laboratory, Upton, NY

11973, USA

Abstract: The reaction p_p ! t_bX is found to be rather rich in exhibiting several different types of CP asymmetries. The spin of the top quark plays an important role. Asymmetries are related to form factors arising from radiative corrections of the tbW production vertex due to non-standard physics. As illustrations, effects are studied in two Higgs Doublet Models and in Supersymmetric Models; asymmetries up to a few % may be possible.

The origin of CP violation remains a pressing issue in Particle Physics. The Standard Model (SM), with three generations of quarks, can accommodate a CP violating phase, the Cabibbo-Kobayashi-Maskawa (CKM) phase [1]. However it is widely believed that this phase cannot account for baryogenesis [2]. Additional CP violating phases due to new physics are therefore a necessity. Besides, in extensions of the SM, new phase(s) appear rather readily. It is therefore quite unlikely that the CKM phase is the only CP violating

1

phase in nature. In particular, in top physics the SM causes negligible CP violation effects [3] whereas, in sharp contrast, non-standard sources often give rise to appreciable effects [4, 5]. Searching for CP violation in top-quark production and/or decay is therefore one of the best ways to look for signals for new physics.

In this paper we examine CP violation asymmetries in top quark production via the basic quark-level reaction:

u + _d ! t + _b; _u + d ! _t + b (1) Indeed the reaction is very rich for CP violation studies as it exhibits many different types of asymmetries. Some of these involve the top spin. Therefore the ability to track the top spin through its decays becomes important and top decays have to be examined as well.

It is easy to see that in the SM these effects are expected to be extremely small as they are severly suppressed by the GIM mechanism [3]. As illustration of the possibilities with non-standard sources of CP violation we consider two extensions of the SM: a two Higgs Doublet Model (2HDM) with Natural Flavor Conservation (NFC), often called type II model and a Supersymmetric Standard model (SSM). We find that CP asymmetries can be sizable, in some cases at the level of a few percent. Thus the asymmetries in t_b production can be appreciably larger than those in t_t pair production [6, 7] wherein they tend to be about a few tenths of percents. Since the number of events needed for observation scales as (asymmetry)\Gamma 2 the enhanced CP violation effects in t_b(_tb) may make up for the reduced production rates for t_b compared to t_t. In fact larger asymmetries are not just gratifying but they can also be essential as detector systematics can be a serious limitation for asymmetries ,! 0:1%.

Let us first discuss the asymmetries in the u _d (_ud) subprocess. We consider four types of asymmetries that may be present. First is the CP violating asymmetry in the cross section:

A0 = (oeq \Gamma oe_q)=(oeq + oe_q) (2) where oeq and oe_q are the cross-sections for u _d ! t_b and _ud ! _tb respectively at ^s = (pt + p_b)2. The CPT theorem of quantum field theory implies that the total cross-section for u _d and d_u are identical. If a cross-section asymmetry

2

A0 in the t_b final state is present, then to maintain the balance of total cross sections, another mode must have a compensating asymmetry.

The spin of the top allows three additional types of CP violating polarization asymmetries. To define these let us introduce the co-ordinate system in the top-quark rest frame where the unit vectors are given by ~ez / \Gamma ~Pb, ~ey / ~Pu \Theta ~Pb and ~ex = ~ey \Theta ~ez. Here ~Pb and ~Pu are the 3-momenta of the_

b quark and the initial u-quark in that frame. We denote the longitudinal polarization or helicity asymmetry as

A(^z) = (NR \Gamma NL \Gamma _NL + _NR)=(NR + NL + _NL + _NR) (3) where NL is the number of left-handed top quarks produced in u _d ! t_b and_ NR is the number of right-handed _t produced in _ud ! _tb etc. In the frame introduced above therefore right handed tops have spin up along the z-axis and left handed ones spin down. We further define the CP violating spin asymmetries in the x and y directions as follows:

A(^x) = (Nx+ \Gamma Nx\Gamma + _Nx+ \Gamma _Nx\Gamma )=(Nx+ + Nx\Gamma + _Nx+ + _Nx\Gamma ) (4)

A(^y) = (Ny+ \Gamma Ny\Gamma \Gamma _Ny+ + _Ny\Gamma )=(Ny+ + Ny\Gamma + _Ny+ + _Ny\Gamma ) (5)

where Nj+( _Nj+) represent the number of t(_t) with spin up with respect to ^j axis, for j = x, y.

While all these four asymmetries are manifestly CP-violating, A0, A(^z) and A(^x) are even under naive time reversal (TN ) whereas A(^y) is TN -odd. So the first three require a complex Feynman amplitude whereas A(^y) needs a real amplitude. Of course, all four do need a CP-violating phase in the underlying theory.

In the limit of massless u and d quarks the CP violating contribution to the W tb vertex may be represented by the effective interaction:

L = i2

\Gamma 12 gW W +

_ _t hF fl

_ + im\Gamma 1

t Goe

ff_qffi L b

\Gamma i2

\Gamma 12 gW W \Gamma

_ _b R hF fl

_ + im\Gamma 1

t Goe

ff_qffi t (6)

where L = (1 \Gamma fl5)=2, R = (1 + fl5)=2, we have taken the b-quark to be massless and consider only the left handed component which will interfere with

3

the standard model (tree-level) contribution. The asymmetries of interest are therefore expressible in terms of F and G alone. We will denote the real and imaginary parts by F = FR + iFI and G = GR + iGI . Note that the terms proportional to FR and GR are hermitian while the terms proportional to FI and GI are not and therefore proportional to final state interaction effects.

We can now derive expressions for the asymmetries in terms of the form factors, for example:

A0 = \Gamma 2FI + 62 + x GI ; A(^z) = 2 2 \Gamma x2 + x FI \Gamma 22 + x GI (7) where x = m2t =^s. Notice that the three quantities fA0; A(^z); A(^x)g are linear combinations of the two form factors fFI; GI g, so in particular, one can show that

A(^x) = \Gamma 3ssx

\Gamma 12 ((2 + x)A

0 + (2 \Gamma x)A(^z)) =32 (8)

The dependence of A0 on FI and GI provides a clue as to how the balance of total cross-section required by CPT is achieved. In order for these imaginary parts to exist in perturbation theory, there must be a contribution from a loop graph which has an intermediate state J that can be kinematically on shell. J is therefore another component of the cross-section, and in fact it is the cross-section asymmetry in J that compensates for the asymmetry in t_b.

The asymmetry A(^y), on the other hand, is proportional to GR. Since GR may be related to GI through a dispersion relation (as a function of ^s) we can obtain A(^y) in terms of A0 and A(^z):

A(^y)[^s] = \Gamma 332 1 \Gamma x(2 + x)px ! ReZ

1

0

2, + x (, \Gamma x)(, \Gamma 1 + iffl), \Delta

\Delta ((2, \Gamma x)A0[,^s] + (2, + x)A(^z)[,^s])d,! (9)

Note that the integrand is 0 if ,^s is below the threshold to produce an imaginary part. In deriving this relation, we assume that some GIM like cancelation applies in the underlying model in which case all of the asymmetries vanish in the limit of large ^s.

4

Fig. 1a shows the SM tree-level production process. The necessary absorptive parts require radiative corrections, involving a CP violating phase, at least to one-loop order. Fig. 1b shows the only graph relevant to a type-II 2HDM with a CP phase residing in neutral Higgs exchanges. Fig. 1c shows example of a one-loop graph that pertains to a SSM which can involve new CP violating phase(s) as well as the needed absorptive parts.

As is well known, in a 2HDM with NFC, CP nonconservation emanates from soft symmetry breaking complex parameters in the Higgs potential [8, 9]. These induce mixing between real and imaginary parts of the Higgs fields in their mass matrix. Consequently the mass eigenstates do not have a definite CP property. Therefore, an important feature of the 2HDM is that CP-violation may result from the neutral Higgs sector even when there is none in the charged Higgs sector. This CP violating phase from neutral Higgs exchanges is much more difficult to look for compared to that from the charged Higgs exchanges. The top quark can play a special role with regard to the neutral Higgs CP as due to its large mass its coupling with the Higgs are significantly enhanced compared to all the other quarks.

A distinctive manifestation of such CP violation is that the neutral Higgs mass eigenstates couple to fermions with both scalar and pseudoscalar couplings. The relevant Feynman rules for calculating the asymmetries of interest can be extracted from the following part of the Lagrangian,

LH0j = H0j _f (afj + ibfjfl5)f + cW jmW H0j g_* W _W * (10) This involves the _f f H0j and the W W H0j couplings, where j = 1; 2; 3 for the three neutral spin 0 fields. The coupling constants, afj, bfj and cW j are functions of tan fi, which is the ratio between the two vacuum expectation values (VEVs) in this model, i.e. tan fi = v2=v1, and of the three mixing angles ff1:::3 which diagonalize the 3 \Theta 3 Higgs mass matrix.

For simplicity we assume that two of the three neutral Higgs particles are much heavier compared to the third one. The effects we are seeking are therefore likely to be dominated by the lightest neutral Higgs. We thus omit the index j in Eqn. 10 and denote the couplings of the lightest Higgs with the top and the W as at, bt and cW .

From Fig. 1 we see that the imaginary part of the loop is provided by the W H intermediate state and hence the cross-section asymmetry A0 is compensated by an asymmetry in u _d ! W +H versus _ud ! W \Gamma H. Clearly

5

this imaginary part can only exist above the W H threshold at ^s = (mW + mH)2. Thus below this A0, A(^z) and A(^x) will be identically 0 though A(^y) need not be since it depends only on virtual effects.

Using the Lagrangian (10) the CP asymmetries, A0 and A(^z), resulting from the interference of Fig. 1a and 1b can be readily calculated:

A0 = \Gamma btcW mW R

0

16ssmt ae(1 \Gamma 3y \Gamma z)OE \Gamma 2(1 \Gamma y)(x + xy \Gamma xz \Gamma 4y)o/ oe (11)

A(^z) = btcW mW R

0

16ssmt(1 \Gamma x) ((1 + 3x \Gamma 7y \Gamma z + 3xy + xz)OE \Gamma 2o/ ^(x \Gamma 2y)

2

+(3x \Gamma 4y)(1 \Gamma z + xz) + x(1 \Gamma x)y(y \Gamma z)*) (12) where x = m2t =^s, y = m2W =^s, z = m2H =^s, OE = p1 + y2 + z2 \Gamma 2y \Gamma 2z \Gamma 2yz, R0 = x=(y(2 + x)(1 \Gamma x)) and

o/ = (1 \Gamma x)

\Gamma 1tanh\Gamma 1[(1 \Gamma x)OE=(1 + x \Gamma y \Gamma z + xz \Gamma xy)] (13)

As should be expected from Eqn. (10) all the CP asymmetries have to be proportional to the product btcW . We choose the angles in the Higgs mixing matrix as ff1 = ff2 = ss=2 and ff3 = 0 as it tends to give maximal effects[10, 11]. Then one can show that btcW mW ' :2mt cos fi cot fi and the asymmetries can now be expressed as a function of tan fi and mH only.

We present our numerical results for tan fi = 0:3 [12]. Numbers for other values of tan fi can then be readily obtained using the relation given in the preceding paragraph. Fig. 2 a and b show the asymmetries as a function of

^s for mH = 100 and 400 GeV. The cross section asymmetry, A0, and helicity asymmetry, A(^z) are typically a few percent. The polarization asymmetry A(^x) likewise is a few percent.

Since the real part of the graph in Fig. 1b does not need a physical threshold, it may receive contributions from Higgs bosons of arbitrary mass. In the limit of degenerate Higgs masses, CP violating effects should vanish. Hence it may not be a legitimate approximation to ignore the more massive Higgs bosons in this loop. We will assume therefore that the other Higgs bosons of the theory have a mass m

0

H and, for the purposes of numericalestimates we will take m 0 H = 1T eV . The effect of these heavy Higgs bosons

6

on the dispersion Eqn. (9) is to replace fA0; A(^z)g by fA00; A0(^z)g which are defined by subtracting the effects of the heavy Higgs. So, for instance, A0 gets replaced by A

0

0[^s] = A0[^s; mH] \Gamma A0[^s; m

0 H] and likewise for the otherasymmetries. Note that if ^s ! (m 0 H + mW )

2 then A0

0 = A0; A

0(^z; ^x) = A(^z; ^x)

since ^s is below the W , H0 threshold. In Fig. 2a and 2b we also show the value of A(^y). There tends to be a cusp at the HW threshold where the peak value of A(^y) is about one percent.

To search for the effects of these three types of spin asymmetries that occur at the production vertex, decays of the top will obviously need to be examined [13]. In particular when considering A(^y) one must keep in mind that it is dependent on the real part of the loop amplitude of Fig. 1b. One complication that this could lead to is that a similar asymmetry may also enter into the decay t ! bW when similar radiative corrections to that vertex are also included [15]. This is not a concern in the case of the other observables since if we assume that the Higgs is above the threshold, i.e. (mW + mH) ? mt, the necessary condition that there be an imaginary part in the decay amplitude is not satisfied.

As it turns out, the observed value of A(^y) is not affected by CP violation in the decay process. The key point is that the measurement of A(^y) through the decay chain u(pu) _d(pd) ! _b(pb1) t(pt) followed by t(pt) ! b(pb2) e+(pe) *(p* ) is equivalent to measurement of the term proportional to ffl(pe; pd; pt; pb1). On the other hand, CP violation arising from the decay process is proportional to ffl(pe; pd; pt; pb2). It is easy to see that an observable related to the first of these will be insensitive to the second[10].

These quark level asymmetries can be readily converted to the hadron (i.e. p_p) level by folding in the structure functions in the standard manner[16]. The results are shown in Fig. 3 where for each of the asymmetries we apply a cut of ^s ? (mH + mW )2. At the Tevatron (E = 2 TeV) the expected number of events are 900-3000 with an integrated luminosity 3f b

\Gamma 1-10f b\Gamma 1

respectively[17]. If the collider energy gets upgraded to 4 TeV and/or there are additional luminosity upgrades as have often been discussed, then the number of events can go up by another factor of about 2 to 10[17]. Thus the asymmetries, in the range of a few percent, resulting from some extensions of the SM may well become within the reach of experiment provided, of course, that the signal for these single top events could be extracted from possible backgrounds[18].

Another extension of the standard model which can produce these kind

7

of asymmetries is SSM. There are a number of possible graphs which could contribute[10, 11]; here we will consider only the gluino exchange diagram given in Fig. 1c. In this case CP violation arises through the mixing matrix between the fermion and the scalar states, in general a 6 \Theta 6 matrix. For simplicity, let us consider a scenario, motivated by supergravity models, where all the squarks are degenerate with a mass ~mq except for the super partner of the top quark, the stop. Furthermore, the two stop states mix with the left and right parts of the top quark with a general 2 \Theta 2 unitary mixing matrixX

. The cross-section asymmetry therefore is given by:

A0 = 2ffsIm(X11X

\Lambda

12) [g( ~mt1; ~mb; ~mg; s) \Gamma g( ~mt2; ~mb; ~mg; s)] (14)

where ~mt1;2 are the masses of the two stops. The function g is given by:

g = (4=3)pxw(1 \Gamma x)

\Gamma 1(2 + x)\Gamma 1OE(Q\Gamma 1tanh\Gamma 1Q \Gamma 1) (15)

where x = m2t =s, y = ~m2q=s, z = ~m2t =s and w = ~m2g=s while OE has the same definitions as previously in terms of y and z and Q = (1 \Gamma x)OE=(1 + 2w \Gamma x \Gamma y \Gamma z + xz \Gamma xy)

In this case the helicity structure of the model is such that the form factor F = G. Thus A(^z) = A0 from which A(^x) and A(^y) follow through Eqn. (8,9). In Fig. 2 we also show these asymmetries due to the SSM for ~mt1 = 100 GeV, ~mt2 = 500 GeV, ~mg = 100 GeV and ~mq = 100 GeV. We have also assumed that the the quantity Im(X11X \Lambda 22) = 1=2 which is its maximum value. We can see that in this case the asymmetries are less than 1%. The small size of these asymmetries is, in part, due to the fact that the intermediate state (see Fig. 1c) consists of two scalars that must be in a P-wave giving rise to an additional threshold suppression factor. However, in SSM many other types of loop corrections (e.g. box graphs) can also contribute giving rise to asymmetries on the order of several percents. [10, 11].

We close with a few remarks in brief.

1. It is important to note that from the point of view of experimental

detection these four asymmetries are independent. Thus, the sensitivity of a given detector to observing the combined CP violation effects may be appreciably better than that for any one asymmetry[10, 11].

8

2. We have focussed here on a p_p machine (i.e. the Tevatron) as the selfconjugate nature of the intial state is rather important for CP studies. At the LHC ( i.e. a pp machine), although the event rate is high, such CP studies are quite difficult. Note, for instance, that the cross-sections for pp ! t_bX and to _tbX are expected to be different at the LHC even if CP was strictly conserved.

3. We recall that the W -glue fusion subprocess, W + + gluon ! t + _b,

also contributes to the same final state[17]. While it will be useful to include its contribution to the asymmetries in a future study, for now we note that, at least in the 2HDM, CP violating radiative corrections, to one loop order, to W g fusion do not yield absorptive parts (in the mb = 0 limit).

We will address to some of these issues in greater detail in future work [10].

This research was supported in part by the US-Israel Binational Science Foundation and in part by USDOE contracts DC-AC05- 84ER40150 (CEBAF) and DE-AC-76CH00016 (BNL).

References

[1] M. Kobayashi and T. Maskawa, Prog. Theor. Phys. 49, 652 (73). [2] For a review see: A.G. Cohen, D.B. Kaplan and A.E. Nelson, Ann. Rev.

Nucl. Part. Phys. 43, 27 (93).

[3] See, e.g., G. Eilam, J. Hewett and A. Soni, Phys. Rev. Lett. 67, 1979

(91); B. Grzadkowski, Phys. Lett B319, 526(93).

[4] See, e.g., C.R. Schmidt and M.E. Peskin, Phys. Rev. Lett. 69, 410 (92);

C.R. Schmidt, Phys. Lett. B293, 111 (92); G.L. Kane, G.A. Ladinsky and C.P. Yuan, Phys. Rev. D45, 124 (92); W. Bernreuther and A. Brandenburg, Phys. Lett. B314, 104 (93); B. Grzadkowski and Y.-Y. Keung, Phys. Lett. B319, 526 (93); E. Christova and M. Fabbrichesi, Phys. Lett. B320, 299 (94); D. Atwood, G. Eilam and A. Soni, Phys. Rev.

9

Lett. 71, 492 (93); R. Cruz, B. Grzadkowski and J.F. Gunion, Phys. Lett. B289 440 (92); B. Grzadkowski, Phys. Lett. B338, 71 (94) and S. Bar-Shalom et al., Phys. Rev. D53, 1162(96).

[5] For a review see, "CP Violation in Top-quark Physics", D. Atwood, S.

Bar-Shalom, G. Eilam and A. Soni, in preparation, for Phys. Reports.

[6] See Schmidt and Peskin, Ref. 4. [7] See Schmidt, Ref. 4; Bernreuther and Brandenburg, Ref. 4. [8] See, e.g., W. Bernreuther, T. Schr"oeder and T.N. Pham, Phys. Lett

B279, 389 (92).

[9] C.D. Froggat, R.G. Moorhouse and I.G. Knowles, Nucl. Phys. B386, 63

(92).

[10] For further details see Bar-Shalom et al., to be published. [11] See also, Shaouly Bar-Shalom, Ph.D Thesis, "Heavy Quark Physics and

CP Violation", Technion (96).

[12] The minimum value, tan fi = 0:3, is adopted following W. Bernreuther

and P. Overmann, preprint .

[13] See, for example, D. Atwood and A. Soni, Phys. Rev. D52, 6271(95);

see also Ref. 14.

[14] B Grzadkowski and J. F. Gunion, Phys. Lett. B350, 218(95). [15] For a discussion of the decay asymmetry see: B. Grzadkowski and J.

Gunion, Phys. Lett. B287, 237 (92).

[16] See, e.g., V. Barger and R. Phillips, "Collider Physics", (AddisonWesley, New York, 1987).

[17] See., e.g., D. Amidei et al., FERMILAB-PUB-96/082; A.P. Heinson,

A.S. Belyaev and E.E. Boos, preprint ; T. Stelzer and S. Willenbrock, Phys. Lett. B357, 125 (95); D.O. Carlson and C.P. Yuan, preprint .

10

[18] We note that experimental issues such as detector efficiencies, backgrounds and the like are clearly very important on their own right, requiring extensive studies that are beyond the scope of this work.

11 Figure Captions Figure 1: Feynman diagrams for contributions to u _d ! t_b. 1a) the standard model process, 1b) one-loop graph in the two Higgs doublet models (2HDM), and 1c) an example of one-loop graph that could occur in the SSM.

Figure 2: The magnitudes of the quark-level asymmetries A0 (solid); A(^z) (dashed); A(^x) (dotted) and A(^y) (dot-dashed) as a function of p^s in the (2HDM) with tan fi = 0:3. Fig. 2a: mH = 100 GeV and Fig. 2b: mH = 400 GeV. Note that A(^y) is computed keeping fixed the masses of the two heavier neutral H0's to 1 TeV. Also shown with the lower solid line is the asymmetry A0 in the SSM described in the text for parameters ~mt1 = 100GeV , ~mt2 = 500GeV , ~mq = 100GeV , ~mg = 100GeV and Im(X11X \Lambda 22) = 1=2.

Figure 3: The corresponding asymmetries in the p_p c.m., for E = 2 TeV, frame as a function of mH for the 2HDM and as a function of \Delta ~mt(j ~mt2 \Gamma

~mt1) for the SSM case. See also caption for Fig. 2.

12

t W

b

u d

a

W u

d

t

b H 0

t

t

b

i

j

g u

d

W

t

b cb

W

Fig.1

13 14 15 16

