

 26 Jul 95

IFT-95-11 DFTT 45/95 hep-ph/yymmnnn

July 1995

THREE-BODY DECAYS OF THE SUSY HIGGS BOSONS\Lambda

Jan Kalinowski

Institute of Theoretical Physics, Warsaw University, ul. Ho.za 69

00681 Warsaw, Poland E-mail: kalino@fuw.edu.pl

and Dipartimento di Fisica Teorica, Universit`a di Torino, via P. Giuria 1

10125 Torino, Italy

ABSTRACT In this talk I discuss three-body decays of the SUSY Higgs bosons which proceed via intermediate two-body states in which one of the particles is off-mass-shell. Some of these decays will be shown to have substantial branching ratios in some range of the SUSY parameter space.

1. Introduction

The Higgs sector of the standard model (SM) still remains to be tested experimentally. Much effort has been devoted to Higgs boson searches in the past years and it remains to be one of the major physics goals of present and future experiments at high-energy colliders.1 Not only we will have to find the Higgs boson(s) and measure its mass but it will be extremely important to measure its couplings to other particles. This will provide a testing ground for the idea of spontaneous symmetry breaking mechanism and to reveal the nature of the underlying theory as being the minimal standard model or one of its possible extensions, like for example supersymmetry.

From the extensive studies of the Higgs sector we have learned that almost all Higgs boson production channels and decay modes are plagued by large SM backgrounds. Therefore it is important to have precise calculations of the production cross sections and decay rates. Since the Higgs boson couplings to SM particles are proportional to their masses (modulo enhancement/suppression factors in extended

\Lambda Presented at the International Symposium and Workshops on Particle Theory and Phenomenology,

May 17-26, 1995, Ames (Iowa).

1

models), the important decay modes are two-body decays to heaviest pair of particles allowed by the phase space. However, in the SM we have also learned that there are important cases when the so called below-threshold, or 3-body (or even 4-body), decay modes are in some cases also very important. A well known example2 is the decay to gauge bosons HSM ! V (\Lambda )V (\Lambda ), (V = W; Z) below the threshold (MHSM ! 2MV ) with one (or two) V 's being off-shell and decaying to fermions. Although suppressed by the off-shell propagator(s), this process is enhanced by the strong Higgs coupling to gauge bosons leading to appreciable branching ratios. In fact, the below-threshold 3-body decay MHSM ! ZZ\Lambda ! Zll (! 4l) provides a very important signature for the intermediate-mass Higgs boson with mass in the range , 130GeV ! MH ! 2MZ.

In the minimal supersymmetric extension of the standard model (MSSM) the decay pattern of Higgs bosons is much richer and 3-body decays may be even more important phenomenologically. In this talk I will discuss various three-body decay modes of the neutral and charged Higgs bosons in the MSSM. To obtain the branching ratios for these processes exact matrix-element calculations for the complete process have been performed, for example H ! W + f _f 0 below W W and H ! t W \Gamma _b below t_t thresholds, or H+ ! _b W +b below t_b threshold. The results presented here have been obtained in collaboration with A. Djouadi and P. Zerwas and details of the calculations can be found in Ref.3 Similar study has been performed in a recent paper by Moretti and Stirling.4

2. Three-body decays in the MSSM

In minimal supersymmetric standard model the Higgs sector has a rich structure.5 There are 5 physical Higgs bosons: two CP-even neutral (h, H, with Mh ! MH ), one CP-odd neutral (A) and a pair of charged (H\Sigma ). Besides their masses, two additional parameters define the properties of these particles: a mixing angle ff in the neutral CP-even sector and the ratio of the two vacuum expectation values tan fi, which from GUT restrictions is assumed to be in the range 1 ! tan fi ! Mt=Mb, with the lower or the upper range, i.e. tan fi , 1 or tan fi , Mt=Mb, being favored since they provide an explanation of the large value of the top quark mass.

Supersymmetry leads to several relations among the parameters of the Higgs sector and only two of them are in fact independent, for example MA and tan fi. The relations among parameters are complicated by radiative corrections6 as the top quark mass is large.7 Since we are not interested here in detailed studies of effects of all possible higher-order corrections but only to find out which three-body decay modes are important, we will incorporate these radiative corrections in terms of the parameter

ffl = 3ff2sss2

W c2W sin2 fi

M 4t M 2Z log 1 +

M 2S M 2t ! ; (1)

s2W = 1 \Gamma c2W j sin2 `W , neglecting non-leading effects due to non-zero values of the supersymmetric Higgs mass parameter _ and of the trilinear couplings At and Ab and

2

assuming a universal soft supersymmetry-breaking scale MS . Then the masses of the other Higgs bosons and the mixing angle ff are given by

M 2h;H = 12 hM 2A + M 2Z + ffl \Gamma f(M 2A + M 2Z + ffl)2

\Gamma 4M 2AM 2Z cos2 2fi \Gamma 4ffl(M 2A sin2 fi + M 2Z cos2 fi)g1=2i (2) M 2H\Sigma = M 2A + M 2W (3)

tan 2ff = tan 2fi M

2A + M 2Z

M 2A \Gamma M 2Z + ffl= cos 2fi ; [\Gamma

ss

2 ! ff ! 0] (4)

The Higgs boson couplings to fermions and gauge bosons are functions of the mixing angles.5;8 These couplings, and hence the value of tan fi, determine to a large extent the decay pattern of the supersymmetric Higgs bosons. They turn out to be quite different for large or small tan fi.

For large values of tan fi(* 5) the pattern is relatively simple because of strong enhancement of the Higgs couplings to down-type fermions. The neutral Higgs bosons will decay into b_b and o/ +o/ \Gamma pairs, the branching ratios being approximately equal to , 90% and , 10% respectively. The charged Higgs particles decay into o/ *o/ pairs below and into tb above the top-bottom threshold. Only in two cases the three-body decays turn out to be interesting: (a) in the case of h when its mass is close to its maximal value, and (b) in the case of H when its mass is close to its minimal value; both exceptions occur however in a very limited domain of the parameter space. The respective Higgs bosons (h in (a), and H in (b)) have then (almost) SM couplingsa and like in the SM the three-body decays into W W \Lambda =ZZ\Lambda , with one of W=Z being off-shell occur with an appreciable rate. Since these processes have been extensively studied in the literature we will not discuss the large tan fi case here (for a full discussion we refer to Ref.3).

This simple pattern can become complicated if one allows for the possibility of Higgs boson decays into supersymmetric particles, like charginos/neutralinos and sfermions.8 Since three-body decays are of importance only for relatively light Higgs bosons, these decays will not be considered here.

For small values of tan fi , 1 the decay pattern is much richer due to several factors: (a) the couplings to bottom quarks and o/ leptons are not significantly enhanced compared to their SM values, (b) the couplings to gauge bosons are not very suppressed, and (c) for masses below , 2Mt for the heavy neutral and , Mt for the charged Higgs bosons, the overwhelming decays into top quark channels are not yet available. Therefore a number of interesting three-body decay modes becomes available which we will discuss in some detail below. In Fig.1-3 we present the results for a representative value of tan fi = 1:5 assuming Mt = 175 GeV.

aIn this case the decays into charm quarks and gluons also occur at a level of , 5%

3

0.001

0.01

0.1

1

150 200 250 300 350 400 450

tauhh

bbtt ZZWW

0.001

0.01

0.1

1

140 150 160 170 180 190 200

ggcc AAAZ H W

Fig. 1. The branching ratios for the heavier CP-even Higgs boson as functions of MH for tan fi = 1:5. 2.1 Three-body decays of the lighter CP-even Higgs boson.

Since its mass is smaller than , 100 GeV for tan fi , 1, the three-body decays of the lighter Higgs boson h may play some role only in the upper mass range, Mh ?, 85 GeV, where h has almost SM-like couplings. In this small mass window the branching rate of the lighter Higgs decay into W W \Lambda can reach the level of a few permille exceeding the physically interesting h ! flfl mode which has a branching ratio of O(10\Gamma 3). Note that the decay h ! AA occurs only for very small values of MA, which are already excluded by LEP data9.

2.2 Three-body decays of the heavier CP-even Higgs boson

For masses below the t_t threshold, the W W=ZZ couplings of the heavier CP-even scalar H are not very suppressed for tan fi , 1 and the decays H ! W W \Lambda =ZZ\Lambda below the 2MW;Z threshold occur with an appreciable rate, see Fig.1. The decays into two pseudoscalar Higgs bosons H ! AA are restricted to a very small domain in the parameter space and can occur at the level of a few percent, and the onshell decay H ! hh can reach up 50 % below t_t threshold. Allowing for one of the final Higgs bosons to be virtual, that is three-body decays H ! AA\Lambda ! Ab_b or H ! hh\Lambda ! hb_b, dramatically suppresses the rate because for tan fi , 1 the A or h have very small Yukawa couplings to bottom quarks. Another possible three-body decays that we find interesting are the cascade decays H ! AZ\Lambda and H ! H\Sigma W \Upsilon \Lambda . Actually, the channel AZ can be open for a large top quark mass which, through radiative corrections, increases MH to values larger than MZ + MA. Increasing MH, the mass of A also increases and for MH * 140 GeV the decay AZ can proceed only

4

0.001

0.01

0.1

1

100 150 200 250 300 350 400 450

taucc

bbgg hZtt

Fig. 2. The branching ratios for the CP-odd Higgs boson as functions of MA for tan fi = 1:5. via Z boson off-shell, i:e: three-body decay, with the branching ratio O(10\Gamma 1) until the channel W W is available. The H ! H\Sigma W \Upsilon \Lambda channel, which is always below threshold, occurs at the level of 1 %. Finally, a few tens of GeV below the 2Mt threshold, the decay of H into a real and a virtual top quark starts to be substantial. Above the 2Mt threshold, the decay H ! t_t overwhelms all other decays and there is little chance for the effect of three-body decays to be important.

2.3 Three-body decays of the CP-odd Higgs boson

For the pseudoscalar Higgs boson the situation is much more interesting. Since it has no couplings to the vector bosons, three-body decays involving heavy particles have an important effect, as can be seen in Fig.2. In some mass range, the propagator suppression is compensated (at least partly) by the much larger couplings of A to the heavy states compared to the coupling to the b quarks into which A decays dominantly. This happens in the case of the three-body decay, A ! hZ \Lambda ! hf _f below 180 GeV, where the price for Z boson to be off-shell and to decay into light fermions is neutralised by the AZh coupling which is much larger than the Abb coupling. On the other hand, the rate for A ! Zh\Lambda is marginal (like in the case H ! AA\Lambda or H ! hh\Lambda ) because of the extremely small hbb Yukawa coupling for tan fi , 1. The two other possibilities, A ! HZ\Lambda and A ! H\Sigma W \Upsilon \Lambda are kinematically closed since the pseudoscalar A is always lighter than the H and H\Sigma bosons. Finally, the decay A ! t_t with one of the top quarks being virtual is very important for masses not too far from the threshold region, since the virtuality of the top quark is compensated by the ratio , M 2t =M 2b of the squared top/bottom Yukawa couplings.

2.4 Three-body decays of the charged Higgs boson

The possibility of three-body decays for the charged Higgs boson H+ ! t_b ! b_bW + below the top-bottom threshold is even more favorable than in the case of the pseudoscalar A, Fig.3. This is due to the smaller value of the o/ lepton and charm quark masses compared to the bottom mass. Similarly, the H\Sigma W h coupling (which is about the same size as the AZh coupling) is much larger than the H\Sigma o/ *o/ and H\Sigma cs Yukawa couplings, allowing for a sizeable branching ratio for the (off-shell below 160 GeV) decay H\Sigma ! hW \Sigma \Lambda . Here again the rate for H\Sigma ! W h\Lambda is very small for

5

0.001

0.01

0.1

1

100 120 140 160 180 200 220 240

tau nucs

cbtb hWAW

Fig. 3. The branching ratios for the charged Higgs boson as functions of MH\Sigma for tan fi = 1:5. values of tan fi close to unity and the decay H\Sigma ! HW \Sigma \Lambda does not occur for real H bosons. The decay H\Sigma ! AW \Upsilon \Lambda is always off-shell and less phase-space favored than the H\Sigma ! hW \Lambda . Note that the effect of the three-body decays AW \Lambda , hW \Lambda and_ bt\Lambda in the mass range MH\Sigma = 90-140 GeV is to lower the decay rate for the most favorable process o/ * from , 90% to 50-70 %. This may have some impact on the interpretation of the top candidate events in the context of extended models.

3. Summary

We have found that for small values of tan fi the three-body decays of Higgs bosons can be important phenomenologically. In some cases they can reduce significantly the decay rates of dominant two-body processes below their values calculated without taking three-body decays into account.

4. Acknowledgements

I would like to express my thanks to A. Djouadi and P. Zerwas for a fruitful collaboration, and A. Ballestrero for several discussions. This work was supported in part by a grant from the Committee for Scientific Research 2 P302 095 05.

6

5. References

1. See for example Proc. of the Workshop e+e\Gamma Collisions at 500 GeV: The

Physics Potential, Munich, Annecy, Hamburg, ed. P. Zerwas, DESY 92- 123/A/B/C (August 1992), and references therein. 2. A. Grau, G. Pancheri and R.J.N. Phillips, Phys. Lett. B251 (1990) 293. 3. A. Djouadi, J. Kalinowski and P. Zerwas, in preparation. 4. S. Moretti and W.J. Stirling, Phys. Lett. B347 (1995) 291. 5. For a review, see J. Gunion, H. Haber, G. Kane and S. Dawson, The Higgs

Hunter's Guide (Addison-Wesley, Reading 1990). 6. Y. Okada, M. Yamaguchi and T. Yanagida, Prog. Theor. Phys. 85 (1991) 1;

H. Haber and R. Hempfling, Phys. Rev. Lett. 66 (1991) 1815; J. Ellis, G. Ridolfi and F. Zwirner, Phys. Lett. B257 (1991) 83; R. Barbieri, F. Caravaglios and M. Frigeni, Phys. Lett. B258 (1991) 167; A. Yamada, Phys. Lett. B263 (1991) 233. 7. F. Abe et al., CDF Collaboration, Phys. Rev. Lett. 74 (1995) 2626;

S. Abachi et al., D0 Collaboration , Phys. Rev. Lett. 74 (1995) 2632. 8. A. Djouadi, J. Kalinowski and P. Zerwas, Z. Phys. C57 (1993) 565. 9. A. Sopczak, CERN preprint CERN-PPE/95-46 and references therein.

7

