On a Subclass of Analytic Functions Related to a Hyperbola

which are analytic in the open unit diskU = {z ∈ C : |z| < 1}. A function f ∈ A is said to be starlike function of order ρ and convex function of order ρ, respectively, if and only if Re{zf(z)/f(z)} > ρ and Re{1 + (zf󸀠󸀠(z)/f󸀠(z))} > ρ, for 0 ≤ ρ < 1 and for all z ∈ U. By usual notations, we denote these classes of functions by S⋆(ρ) and K(ρ) (0 ≤ ρ < 1), respectively. We write S⋆(0) = S⋆ and K(0) = K, the familiar subclasses of starlike functions and convex functions inU. Furthermore, a function f ∈ A is said to in the class R(ρ), if it satisfies the inequality:

With the aid of the linear operator L(, ), we introduce a subclass of A as follows.
Definition 1.A function  ∈ A is said to be in the class R(, , ), if it satisfies the following subordination relation: where the power in the right hand side of (13) indicates the principal branch.Note that if  ∈ R(, , ), then by ( 13) We denote by R(2, 1, ) = R(), the class of functions  ∈ A satisfying the subordination condition: In fact, by suitably specializing the parameters , , and  in the class R(, , ), we can obtain several subclasses of A.
Remark 2. To bring out the geometrical significance of the class R(, , ), we set and note that which gives   ( which on simplification reduces to is the interior of the right half branch of the hyperbola , where ℎ  is given by (16).
Fekete and Szegö [9] defined the Hankel determinant of a function , given by (1) as In our present investigation, we also consider the second Hankel determinant of , given by It is known [10] that if  given by ( 1) is analytic and univalent in U, then the sharp inequality  2 (1) = | 3 −  2 2 | ≤ 1 holds.For a family F of functions in A of the form (1), the more general problem of finding the sharp upper bounds for the functionals ( 3 −  2  2 ) ( ∈ R/C) is popularly known as Fekete-Szegö problem for the class F. The Fekete-Szegö problem for the known classes of univalent functions, starlike functions, convex functions, and close-to-convex functions has been completely settled [9,[11][12][13][14][15][16][17][18].Recently, Janteng et al. [19,20] have obtained the sharp upper bounds to the second Hankel determinant  2 (2) for the family R of functions in A whose derivatives have positive real part in U.For initial work on the class R, one may refer to the paper by MacGregor [21].
Our objective in the present paper is to solve the Fekete-Szegö problem and also to determine the sharp upper bound to the second Hankel determinant for the class R(, , ) by following the techniques devised by Libera and Złotkiewicz [22,23].The criteria for functions in A to be in this class are also obtained.
To establish our main results, we will need the following results about the functions belonging to the class P. Lemma 3. Let the function , given by (3), be a member of the class P. Then for some complex numbers ,  satisfying || ≤ 1 and || ≤ 1.

Main Results
Unless otherwise mentioned, we assume throughout the sequel that Now, we determine the sharp upper bound for the functional | 3 −  2  2 | ( ∈ C) for functions of the form (1) belonging to the class R(, , ).Theorem 4. Let  > 0 and  > 0. If the function , given by (1), belongs to the class R(, , ), then for any The estimate in (26) is sharp.
Proof.First, we assume that  < −{( + 1)}/2( + 1).Then and by using (26) again, we get ( The estimates are sharp for the function  0 defined in U by where the function ℎ  is given by ( 16) and the proof of Theorem 5 is completed.
Using ( 21) in (29) and putting  = 0 and  = 1, respectively, in Theorem 5, we get the following.Corollary 6.Let  ≥  > 0. If the function , given by (1), belongs to the class R(, , ), then The estimates in (44) and (46) are sharp for the function  0 defined by whereas the estimate in (45) is sharp for the function  0 given by where the function ℎ  is given by (16).
Letting  = 2 and  = 1 in Theorem 8, we obtain the following.

Corollary 7. If the function 𝑓, given by (1), belongs to the class R(𝜌), then
International Journal of Mathematics and Mathematical Sciences 5 The estimates are sharp for the function  0 defined in U by where ℎ  is given by (16).
Next, we find the sharp upper bound for the fourth coefficient of functions in the class R(, , ).Theorem 8. Let the function , given by (1), belong to the class R(, , ).Then and the estimate in (51) is sharp.
Applying the triangle inequality in the above expression followed by the replacement of || with  in the resulting equation, we obtain We next maximize the function (, ) on the closed rectangle we have / < 0 for 0 <  < 2 and 0 <  < 1.Thus, (, ) cannot have a maximum in the interior on the closed rectangle where A routine calculation yields for  = 0 or  = 8/{3(1 +  2 )}.Since   (0) = −1 < 0 and   (8/{3(1 +  2 )}) = 1 > 0, we conclude that the maximum of  is attained at  = 0. Thus, the upper bound of the function  corresponds to  =  = 0. Putting  =  = 0 in (54), we get our desired estimate (51).Equality in (51) holds for the function  0 defined by where ℎ  is given by ( 16).

International Journal of Mathematics and Mathematical Sciences
As in Theorem 8, we assume without loss of generality that  1 > 0 and for convenience of notation, we write  1 =  (0 ≤  ≤ 2).By using ( 23) and ( 24) in (61), we get Since  2 > 4 and by the hypothesis, we conclude that the maximum value of F is attained at  = 0 so that the upper bound of the function G corresponds to  = 0 and  = 1.Thus, by letting  = 0 and  = 1 in (63), we get the estimate (60).The estimate in (60) is sharp for the function  0 given by (48).This completes the proof of Theorem 9.
Putting  = 2 and  = 1 in Theorem 9, we get the following.
Corollary 10.If the function , given by (1) belongs to the class R(), then and the estimate is sharp for the function  0 defined by where the function ℎ  is given by (16).
Proof.We define the function  by Choosing the principal branch in the right hand side in (74), we note that  is analytic in U with (0) = 0. Furthermore, logarithmically differentiating (74) and using the identity (12) in the resulting equation, we find that  (82) and the proof of Theorem 11 is completed.
In the special case  = 2, we get the following sufficient condition for the class R(, , ).