On the Ground State to Hamiltonian Elliptic System with Choquard’s Nonlinear Term

<jats:p>In the present paper, we consider the following Hamiltonian elliptic system with Choquard’s nonlinear term <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M1"><mml:mfenced open="{" close=""><mml:mrow><mml:mtable class="cases"><mml:mtr><mml:mtd columnalign="left"><mml:mo>−</mml:mo><mml:mi>Δ</mml:mi><mml:mi>u</mml:mi><mml:mo>+</mml:mo><mml:mi>V</mml:mi><mml:mfenced open="(" close=")"><mml:mrow><mml:mi>x</mml:mi></mml:mrow></mml:mfenced><mml:mi>u</mml:mi><mml:mo>=</mml:mo><mml:msub><mml:mrow><mml:mo>∫</mml:mo></mml:mrow><mml:mrow><mml:mi>Ω</mml:mi></mml:mrow></mml:msub><mml:mfenced open="(" close=")"><mml:mrow><mml:mi>G</mml:mi><mml:mfenced open="(" close=")"><mml:mrow><mml:mi>v</mml:mi><mml:mfenced open="(" close=")"><mml:mrow><mml:mi>y</mml:mi></mml:mrow></mml:mfenced></mml:mrow></mml:mfenced><mml:mo>/</mml:mo><mml:msup><mml:mrow><mml:mfenced open="|" close="|"><mml:mrow><mml:mi>x</mml:mi><mml:mo>−</mml:mo><mml:mi>y</mml:mi></mml:mrow></mml:mfenced></mml:mrow><mml:mrow><mml:mi>β</mml:mi></mml:mrow></mml:msup></mml:mrow></mml:mfenced><mml:mi>d</mml:mi><mml:mi>y</mml:mi><mml:mi>g</mml:mi><mml:mfenced open="(" close=")"><mml:mrow><mml:mi>v</mml:mi></mml:mrow></mml:mfenced><mml:mtext> </mml:mtext><mml:mtext>in</mml:mtext><mml:mtext> </mml:mtext><mml:mi>Ω</mml:mi><mml:mo>,</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd columnalign="left"><mml:mo>−</mml:mo><mml:mi>Δ</mml:mi><mml:mi>v</mml:mi><mml:mo>+</mml:mo><mml:mi>V</mml:mi><mml:mfenced open="(" close=")"><mml:mrow><mml:mi>x</mml:mi></mml:mrow></mml:mfenced><mml:mi>v</mml:mi><mml:mo>=</mml:mo><mml:msub><mml:mrow><mml:mo>∫</mml:mo></mml:mrow><mml:mrow><mml:mi>Ω</mml:mi></mml:mrow></mml:msub><mml:mfenced open="(" close=")"><mml:mrow><mml:mi>F</mml:mi><mml:mfenced open="(" close=")"><mml:mrow><mml:mi>u</mml:mi><mml:mfenced open="(" close=")"><mml:mrow><mml:mi>y</mml:mi></mml:mrow></mml:mfenced></mml:mrow></mml:mfenced><mml:mo>/</mml:mo><mml:msup><mml:mrow><mml:mfenced open="|" close="|"><mml:mrow><mml:mi>x</mml:mi><mml:mo>−</mml:mo><mml:mi>y</mml:mi></mml:mrow></mml:mfenced></mml:mrow><mml:mrow><mml:mi>α</mml:mi></mml:mrow></mml:msup></mml:mrow></mml:mfenced><mml:mi>d</mml:mi><mml:mi>y</mml:mi><mml:mi>f</mml:mi><mml:mfenced open="(" close=")"><mml:mrow><mml:mi>u</mml:mi></mml:mrow></mml:mfenced><mml:mtext> </mml:mtext><mml:mtext>in</mml:mtext><mml:mtext> </mml:mtext><mml:mi>Ω</mml:mi><mml:mo>,</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd columnalign="left"><mml:mi>u</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo><mml:mi>v</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn><mml:mtext> </mml:mtext><mml:mtext>on</mml:mtext><mml:mtext> </mml:mtext><mml:mi>∂</mml:mi><mml:mi>Ω</mml:mi><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:mrow></mml:mfenced></mml:math>where <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M2"><mml:mi>Ω</mml:mi><mml:mo>⊂</mml:mo><mml:msup><mml:mrow><mml:mi>ℝ</mml:mi></mml:mrow><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:msup></mml:math> is a bounded domain with a smooth boundary, <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M3"><mml:mn>0</mml:mn><mml:mo><</mml:mo><mml:mi>α</mml:mi><mml:mo><</mml:mo><mml:mi>N</mml:mi></mml:math>, <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M4"><mml:mn>0</mml:mn><mml:mo><</mml:mo><mml:mi>β</mml:mi><mml:mo><</mml:mo><mml:mi>N</mml:mi></mml:math>, and <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M5"><mml:mi>F</mml:mi></mml:math> is the primitive of <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M6"><mml:mi>f</mml:mi></mml:math>, similarly for <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M7"><mml:mi>G</mml:mi></mml:math>. By establishing a strongly indefinite variational setting, we prove that the above problem has a ground state solution.</jats:p>


Introduction and Main Results
In this paper, we deal with the existence of ground state solutions for the following Hamiltonian elliptic system with Choquard's nonlinear term: where Ω ⊂ ℝ N is a bounded domain with a smooth boundary, N ≥ 3, 0 < α < N, 0 < β < N, and F is the primitive of f , similarly for G. For a single equation in whole space, ℝ N is closely related to the Choquard-Pekar equation: When N = 3, μ = 1, VðxÞ ≡ 1, and f ðuÞ = u, equation (2) has appeared in several contexts of quantum physics. In 1954, Pekar used equation (2) to describe a polaron at rest in quantum theory. In 1976, to model an electron trapped in its own hole, P. Choquard considered (2) as a certain approximation to Hartree-Fock's theory of one component plasma (see [1,2]). In some particular cases, (2) is also known as the Schrödinger-Newton equation which was introduced by Penrose in [3] to describe the self-gravitational collapse of a quantum mechanical wave function.
Motivated by the papers mentioned above, in particular, the papers [13,14,17], the purpose of the present paper is to investigate the ground state solution of problem (1). To the best of our knowledge, there is no work concerning the existence of ground state solutions to the Choquard-type Hamiltonian elliptic system. For the Hamiltonian elliptic system, we refer the readers to the papers [18][19][20][21][22] and the references therein.
Throughout this paper, we will always assume N ≥ 3 and we suppose that V, g, and f satisfy the following assumptions. where σð−Δ + VðxÞÞ is the spectrum of the operator −Δ + VðxÞ.
Before stating our main result, we review the definition of ðs, tÞ ground state solution about (1). We call ðu, vÞ ≠ ð0, 0Þ as a ðs, tÞ ground state solution of (1) in work space E (see Section 2); if ðũ,ṽÞ is another ðs, tÞ weak solution (see Definition 2), then the corresponding energy functional Iðu, vÞ ≤ Iðũ,ṽÞ, where I will be defined in equation (27)  To prove Theorem 1, here we use a minimizing argument based on the ideas developed in [23]. We are concerned with system (1) in whole space ℝ N involving the Hamiltonian elliptic system with Choquard's nonlinear term; it is a nonlocal problem, which brings about two obstacles. One is to check the linking structure, and the other one is to prove the boundedness of the corresponding (PS) sequences. To avoid these obstacles, we shall deal with our problem in bounded domain. Indeed, the difficulty is still there for the problem on ℝ N . It is worth pointing out that the monotonicity condition like [23] is not required on the nonlinear terms f and g; it prevents us from using the standard way (see, e.g., [23,24]) to check that the minimizer is a critical point. Via the basic leitmotiv from Proposition 3.2 of [25], we shall use the deformation lemma to prove it (see Lemma 13.. We end this section by giving our arrangements of this paper. In Section 2, we establish the variational settings about (1). In Section 3, we provide some lemmas and then prove Theorem 1.

Variational Settings
For the system is well defined if FðuÞ ∈ L r ðΩÞ for t > 1 such that ð2/rÞ + ðα/ NÞ = 2. In view of Sobolev's embedding theorem, it requires that 1 ≤ ðθ + 1Þr ≤ 2N/ðN − 2Þ, which leads us to assume that Since we restrict p ≥ 1, q ≥ 1 and So it holds that 1 − ðα/2NÞ ≤ p + 1 and 1 − ðβ/2NÞ ≤ q + 1 but p or q could be supercritical in the sense that We remark that p ≥ 1 is used in (38) and similarly for q ≥ 1. Since p or q could be supercritical, we need the fractional Sobolev spaces (see, e.g., [19,22,27]). According to H 4 , we can choose s, t > 0 with s + t = 2, such that Denote S = −Δ + VðxÞ. Since the effective domain DðSÞ = DðS * Þ and S is symmetric, so S is self-adjoined on L 2 ðΩÞ. Since the self-adjoint operator is closed, according to the polar decomposition theorem (Theorem VIII.32 in [28] and jointly with Theorem 3.2 and 3.3, Ch IV in [29]), it holds that there is a positive self-adjoint operator jSj (in fact jSj = ffiffiffiffiffiffiffi S * S p ), with DðjSjÞ = DðSÞ and a partial isometry U such that S = UjSj, jSj and U are uniquely determined and It is well known that U and jSj are both self-adjoint operators on L 2 ðΩÞ. In view of Theorem 3.35 in Chapter V of [30] and Corollary 5.5.6 in [31], there is a unique square root operator Q such that Q 2 = jSj, furthermore, QU = UQ: Denote jSj 1/2 ≔ Q: Let According to [19,27], we consider a basis of L 2 ðΩÞ constituted by eigenfunctions fϕ n g of with associated eigenvalues λ n . If for u ∈ L 2 ðΩÞ, we write u = ∑ ∞ n=1 a n ϕ n , then the effective domain of A s and A t are They are two Hilbert spaces endowed with the following inner product, respectively, where is an isometric isomorphism. So, A s has the inverse ðA s Þ −1 , and we denote A −s = ðA s Þ −1 , similarly for A t . Set E = E s × E t , then E is a Hilbert space with the inner product and norm for z = ðu, vÞ, η = ðφ, ψÞ ∈ E. We recall the embedding theorem (see [32]); for r > 0, the embedding E r ⟶ L k ðℝ N Þ is continuous for 1 ≤ k ≤ ð2N/ðN − 2rÞÞ and is compact for 1 ≤ k < ð2N/ðN − 2rÞÞ. We also consider the bounded selfadjoint operator L : E ⟶ E defined as follows, for z = ðu, vÞ, η = ðφ, ψÞ: A natural question will be asked whether the operator L : E ⟶ E is well defined. Here, we only check if hA s Uu, A t ψi L 2 is well defined. In fact, noting that A s and U are self-adjoint operators, we infer that

Advances in Mathematical Physics
By the complex interpolation theory (see Chapter 1.15 of [32]), we get So hA s Uu, A t ψi L 2 < ∞. Next, we have L that has only two eigenvalues −1 and 1, whose corresponding eigenspaces are Clearly, E = E + ⊕ E − . Indeed, 0 lies in a gap of σðð?+ VðxÞÞÞ, and using Theorem 3.3 in Chapter IV of [29], we get where L ± = u ∈ L 2 ðΩÞ: Uu = ±u.
Similar to Proposition 1.1 in [19], for z = ðu, vÞ, we can easily get We consider the eigenvalue problem Lz = λz in E, which yields that Here, we have used the fact that A s U = UA s and A t U = UA t (see [22]).
We consider the corresponding energy functional for (7): Similarly ( [27], p. 61), under our assumptions, using the fact that A s U and A t are linear, jointly with Lebesgue's dominated convergence theorem, the Gateaux derivative of I in the direction η = ðφ, ψÞ at z = ðu, vÞ is defined as Clearly, DIðz, ηÞ is linear bounded about η and continuous about z. Thus, I ∈ C 1 ðE, ℝÞ. And its Fréchet derivative is given by Definition 2. We say that z = ðu, vÞ ∈ E \ f0g is a ðs, tÞ weak solution of (7), if for each η = ðφ, ψÞ ∈ E, there holds

Advances in Mathematical Physics
It is easy to check that the critical point ðu, vÞ of I is a ðs, tÞ weak solution of (7). Moreover, for z = ðu, vÞ ∈ E, z = z + + z − , in view of Lemma 2.1 in [19], it holds that Therefore, Remark 3. If inf x∈Ω VðxÞ > 0, then U = I.

Existence of Ground States
Following [23] (in page 3804), we introduce the generalized Nehari manifold We need to prove that M ≠ ∅.
Proof. By Lemma 4, it suffices to prove that the maximum exists. This follows from the next two lemmas. Indeed, using Lemma 7 and Lemma 8, combining with Ið0Þ = 0, it is easy to check that the maximum point exists.
Remark 6. Even though f and g have satisfied the condition ðS 5 Þ in [23], we cannot have the uniqueness of t z z + + η z . So, we cannot get the result similar to Proposition 2.3 in [23].

Lemma 7.
There are r > 0 and ρ > 0 such that inf Proof. For a given ε > 0, by H 2 , H 3 , and H 4 , there is a CðεÞ > 0 such that By Hardy-Littlewood-Sobolev's inequality, one has ð where ð2/rÞ + ðα/NÞ = 2. Here we use the fact that Similarly, we have ð Thus, for z = ðu, A −t A s UuÞ ∈ E + , it holds that Hence, we can choose some r, ρ > 0 such that IðzÞ ≥ r for all kzk = ρ.
Proof. If not, there exists fw n g ⊂ÊðeÞ such that kw n k ⟶ ∞ and Iðw n Þ ≥ 0.
Without loss of generality, we may assume that e ∈ E + , kek = 1. By doing ðw n,1 , w n,2 Þ = w n = t n e + w − n where t n ≥ 0.
Obviously, 1 = k w n k 2 = s 2 n + kη − n k 2 . Jointly with where jAj > 0. Thus, for x ∈ A, jw n ðxÞj ⟶ ∞. It follows from H 2 , H 3 , and H 5 that there exists C 1 , C 2 > 0 such that and Thus, w n = ðw n,1 , w n,2 Þ, it follows from Fatou's lemma that 1 ð Þ+ G w n,2 ð Þ w n j j 2 w n j j 2 w n k k 2 dx ⟶ ∞, ð46Þ which would conflict with (38). Here, we use the fact that Ω is bounded.

Lemma 9.
M contains all nontrivial critical points.