Production and decay of up-type and down-type new heavy quarks through anomalous interactions at the LHC

We study the process pp->QVX (where Q=t,b and V=g,gamma,Z) through the anomalous interactions of the new heavy quarks at the LHC. Considering the present limits on the masses and mixings, the signatures of the heavy quark anomalous interactions are discussed and analysed at the LHC for the center of mass energy of 13 TeV. An important sensitivity to anomalous couplings kappa_{g}^{t'}/Lambda=0.10 TeV^{-1}, kappa_{\gamma}^{t'}/Lambda=0.14 TeV^{-1}, kappa_{Z}^{t'}/Lambda=0.19 TeV^{-1} and kappa_{g}^{b'}/Lambda=0.15 TeV^{-1}, kappa_{Z}^{b'}/Lambda=0.19 TeV$^{-1}, kappa_{\gamma}^{b'}/Lambda=0.30 TeV$^{-1} for the mass of 750 GeV of the new heavy quarks t' and b' can be reached for an integrated luminosity of L_{int}=100 fb^{-1}.


I. INTRODUCTION
The standard model (SM) of the strong and electroweak interactions describes succesfully the phenomena of particle physics. However, there are many unanswered questions suggesting the SM to be an effective theory. In order to answer some of the problems with the SM, additional new fermions can be accommodated in many models beyond the SM (see Refs. [1], [2], [3], [4] and references therein). The new heavy quarks could also be produced in pairs at the LHC with center of mass energy of 13 TeV. However, due to the expected smallness of the mixing between the new heavy quarks and known quarks, the decay modes can be quite different from the one relevant to charged weak interactions. A new symmetry beyond the SM is expected to explain the smallness of these mixings. The arguments given in Ref. [5] for anomalous interactions of the top quark are more valid for the new heavy quarks t ′ and b ′ due to their expected larger masses than the top quark.
The ATLAS experiment [6] and CMS experiment [7] have searched for the fourth generation of quarks and set limits on the mass of m t ′ > 570 GeV and m b ′ > 470 GeV at √ s = 7 TeV. The pair production of new heavy quarks have been searched by the ATLAS experiment [8], [9] and the m t ′ > 656 GeV mass limits are set at √ s = 7 TeV. The CMS experiment have excluded t ′ masses below 557 GeV [10] . The vector-like quarks have been searched by the ATLAS experiment [11], [12] and set bounds as 900 GeV for charged current channel and 760 GeV for neutral current channel at √ s = 7 TeV. The CMS experiment [13], [14] have set the lower bounds on the mass of 687 GeV at √ s = 8 TeV. Some of the final states in the searches of new phenomena [15] and excited quarks [16] can also be considered in relation with the new heavy quarks.
The anomalous resonant productions of the fourth family quarks have been studied in Refs. [17,18] at the LHC with √ s = 14 TeV. sThe possible single productions of fourth generation quarks via anomalous interactions at Tevatron have also been studied in Refs. [19,20]. The parameter space for the mixing of the fourth generation quarks have been presented in Ref. [21]. The CP violating flavour changing neutral current processes of the fourth generation quarks have been analysed in Ref. [22], and the large mixing between fourth generation and first three generations have been excluded under the proposed fit conditions. Investigation of the parameter space favoured by the precision electroweak data have been performed for the fourth SM family fermions in Ref. [23].
In this work, we present the analysis of anomalous productions and decays of new heavy quarks t ′ and b ′ at the LHC. We have performed the fast simulation for the signal and background. Any observations of the invariant mass peak in the range of 500 − 1000 GeV and excess in the events with the final states originating from tV and bV can be interpreted as the signal for the new heavy quarks t ′ and b ′ via the anomalous interactions.

II. HEAVY QUARKS ANOMALOUS INTERACTIONS
A general theory that has the standard model (SM) as its low energy limit can be written as a series in Λ −1 with operators obeying the required symmetries. The effective Lagrangian for the anomalous interactions among the heavy quarks (Q ′ ≡ t ′ or b ′ ), ordinary quarks q, and the gauge bosons V = γ, Z, g can be written explicitly: where F µν , Z µν and G µν are the field strength tensors of the gauge bosons; σ µν = i(γ µ γ ν − γ ν γ µ )/2; λ a are the Gell-Mann matrices; Q q is the electric charge of the quark (q); g e , g Z and g s are the electromagnetic, neutral weak and the strong coupling constants, respectively. g Z = g e / cos θ w sin θ w , where θ w is the weak mixing angle. κ γ is the anomalous coupling with photon; κ z is for the Z boson, and κ g is the coupling with gluon. Finally, Λ is the cutoff scale for the new interactions.

III. DECAY WIDTHS AND BRANCHINGS
For the decay channels Q ′ → V q where V ≡ γ, Z, g, we use the effective Lagrangian to calculate the anomalous decay widths The anomalous decay widths in different channels are proportional to Λ −2 , and they are assumed to be dominant for κ/Λ > 0.1 TeV −1 over the charged current channels. In this case, if we take all the anomalous coupling equal then the branching ratios will be nearly independent of κ/Λ. We have used three parametrizations sets entitled PI, PII and PIII.
For the PI parametrization, we assume the constant value κ i /Λ = 0.1 TeV −1 , and PII has the parameters κ i /Λ = 0.1λ 4−i TeV −1 with λ = 0.5. For PIII we take the couplings κ i /Λ = 0.5λ 4−i TeV −1 with the same value of λ. The index i is the generation number. Table I and Table II present the decay width and branching ratios of the new heavy quark t ′ through anomalous interactions for the parametrizations PI, PII and PIII, respectively.
Taking the anomalous coupling κ/Λ = 0.1 TeV −1 we calculate the t ′ decay width Γ = 0.65 GeV and 1.90 GeV for m t ′ = 700 GeV and 1000 GeV, respectively. The branching into t ′ → qg channel is the largest and branching into t ′ → qγ channel is the smallest for equal anomalous couplings with the parametrization PI. On the other hand, PII and PIII parametrizations give higher branching ratios into tV (V = g, Z, γ) than qV (q = u, c) channels due to λ 4−i factor in the parametrizations.
For the new heavy quark b ′ the decay witdh and branching ratios are presented in Table   III and Table IV for the parametrizations PI, PII and PIII, respectively. We calculate the b ′ decay width, by taking the anomalous coupling κ/Λ = 0.1 TeV −1 , Γ = 0.68 GeV and 1.92 GeV for m b ′ = 700 GeV and 1000 GeV, respectively. The branching for b ′ → qg is the largest (30%) and its the smallest for b ′ → qγ (0.2%) channel for equal anomalous couplings with the parametrization PI. For PII and PIII parametrizations the branching ratios into  bV (V = g, Z, γ) are larger than qV (q = d, s) channels. The t ′ and b ′ decay widths are about the same values for PII and PIII parametrizations.

IV. THE CROSS SECTIONS
In order to study the new heavy quark productions at the LHC, we have used effective anomalous interaction vertices and implemented these vertices into the CalcHEP package [24]. In all of the numerical calculations, the parton distribution function are set to the CTEQ6L parametrization [25]. The new heavy quarks can be produced through its anomalous couplings to the ordinary quarks and neutral vector bosons as shown in Fig. 1.
Total cross sections for the productions of new heavy quarks t ′ and b ′ are given in Table   V and Table VI for the parametrizations PI, PII and PIII, at the center of mass energy of 8    Table V and Table VI, the cross sections decreases while the mass of the new heavy quark increases. The cross section for t ′ production is larger than the b ′ production with a factor of 1.2-1.8 (0.7-1.0) for PI (PII and PIII) parametrization   depending on the considered mass range at √ s = 13 TeV. The general behaviours of the production cross sections depending on the mass of heavy quarks are presented in Fig. 2 and Fig. 3 for different parametrizations.  A. Analysis of the process pp → W + bV + X (V = g, Z, γ) for t ′ signal The signal process pp → W + bV + X (V = g, Z, γ) includes the t ′ exchange both in the s-channel and t-channel. The s-channel contribution to the signal process would appear itself as resonance around the t ′ mass value in the W bV invariant mass. The t-channel gives the non-resonant contribution. We consider that the W boson decays into lepton+missing transverse momentum with the branching ratio 21% and Z boson decays into dilepton with the branching 6.7%. In our analyses, we consider the t ′ signal in the l + b jet + γ + MET , l + b jet + j + MET and 3l + b jet + MET channels, where l = e, µ. However, if one takes the hadronic W decays the signal will be enhanced by a factor of BR(W → hadrons)/BR(W → lν).
We have obtained the cross sections by using the cuts pseudorapidity |η j,γ | < 2.5 and transverse momentum p j,γ T > 20−200 GeV for jets and photon, in Table VII (TableVIII, Table  IX) for PI (PII, PIII) parametrizations, respectively. It appears from signal significance calculations that the optimized transverse momentum cut is p T >100 GeV for t ′ analyses.
The backgrounds for the final state W + b(b)V (where V ≡ photon, jet and Z boson) are given in Table X. We apply the following cuts to the final state photon and jets as |η j,γ | < 2.5 and p j,γ T > 20 − 200 GeV. For the background cross section estimates, we assume the efficiency for b-tagging to be ε b = 50%, and the rejection ratios 10% for c (c) quark jets and 1% for light quark jets since they are assumed to be mistagged as b-jets.
In order to find the discovery limits we use the statistical significance as where S and B are the numbers of the signal and background events, respectively. In Figs. 4-6, the integrated luminosity required to reach 3σ significance for the signal of t ′ anomalous interactions is shown for parametrization PI, PII and PIII at the LHC with √ s = 13 TeV.
It is seen from these figures that the channel t ′ → tZ requires more integrated luminosity than the other channels. By requiring the signal significance SS = 3, the contour plots of κ/Λ and mass of t ′ quark are presented in Fig. 7. The results show that one can discover the t ′ quark anomalous couplings κ/Λ down to 0.1 TeV −1 in the tg channel for m t ′ =750 GeV.

B. Analysis of the process pp
The signal process pp → bV + X (V = g, Z, γ) includes the new heavy quark b ′ exchange both in the s-channel and t-channel. The s-channel contributes to the signal process as resonance around the b ′ mass value in the bV invariant mass, while the t-channel contributes to the non-resonant behaviour. For this process, we consider the leptonic decays of Z boson.
In the analyses, we consider the b ′ signal to be b jet + γ , b jet + j and b jet + dilepton.
We have obtained the cross sections by using the pseudorapidity cuts |η j,γ | < 2.5 and transverse momentum cuts p j,γ T > 20 − 200 GeV for jets and photon, in Table XI (Table  XII, Table XIII for PI (PII, PIII) parametrizations, respectively. It appears from signal        cuts increases. We assume the efficiency for b-tagging to be ε b = 50%, and the rejection ratios 10% for c (c) quark jets and 1% for light quark jets.
In order to reach 3σ significance for the signal of b ′ anomalous interactions the required integrated luminosity is shown in Figs. 8-10 for parametrizations PI, PII and PIII at the LHC with √ s = 13 TeV. The channel b ′ → bγ requires more integrated luminosity than the other channels. By requiring the signal significance SS = 3, the contour plots of κ/Λ and   Fig. 11. The results show that one can discover the b ′ quark anomalous couplings down to 0.1 in the bg channel for m b ′ =500 GeV.

V. CONCLUSION
The new heavy quarks of up-type and down-type can be produced with large numbers at the LHC if they have the anomalous couplings (via flavour changing neutral current)    TeV. The anomalous vertices could appear significantly at leading order processes due to the possiblity of new heavy quarks. From the results of signal significance calculations for t ′ (b ′ ) anomalous productions, the sensitivity to the anomalous couplings κ t ′ /Λ (κ b ′ /Λ) can be