POSITIVE SOLUTIONS OF CRITICAL QUASILINEAR ELLIPTIC PROBLEMS IN GENERAL DOMAINS

We consider a certain class of quasilinear elliptic equations with a term in the critical growth range. We prove the existence of positive solutions in bounded and unbounded domains. The proofs involve several generalizations of standard variational arguments.


Introduction
We study the existence of positive functions u ∈ D 1,2  0 (Ω) solving in a weak sense the quasilinear elliptic equation where Ω ⊂ IR n (n ≥ 3) is open (not necessarily bounded), D 1,2 0 (Ω) is the completion of the space C ∞ c (Ω) with respect to the Dirichlet norm ( u 2 := Ω |∇u| 2 ), 2 * = 2n n−2 is the critical Sobolev exponent q of the embedding D 1,2 0 ⊂ L q , g is a subcritical term and p a bounded positive function.Throughout this paper by a positive function we mean a nonnegative nontrivial function.Weak solutions of (1) correspond to critical points of the functional J defined for all u ∈ D 1,2  0 (Ω) by ( 2) where G(x, s) = s 0 g(x, t)dt.Under reasonable assumptions on a ij , g, p, the functional J is continuous but not even locally Lipschitz if the functions a ij (x, s) depend on s, see [9].However, J is weakly C ∞ c (Ω)-differentiable (see [3,9]) and the derivative of J exists in the smooth directions: for all u ∈ D 1,2  0 (Ω) and ϕ ∈ C ∞ c (Ω) we can evaluate a ij (x, u)D i uD j ϕ + 1 2 according to the nonsmooth critical point theory of [14,15] it is possible to prove that critical points u of J satisfy J (u)[ϕ] = 0 ∀ϕ ∈ C ∞ c (Ω) and hence solve (1) in distributional sense, see also [2].Therefore, as (here (D 1,2 0 ) * denotes the dual space of D 1,2 0 ) we also have 1 2 and ( 1) is solved in a weak sense.We refer to the original papers [9,14,15] for the basic definitions in this nonsmooth context; this theory has been widely used for different problems related to quasilinear elliptic equations of the kind of (1), see [3,4,9,10,13].
Under suitable assumptions on the functions a ij , g and p, in this paper we prove the existence of positive solutions of (1) in bounded and unbounded domains Ω: making use of the techniques introduced in [21], we prove our results for a wide class of subcritical perturbations g.As far as we are aware, very few results concerning (1) are known: apart the already mentioned case with p(x) ≡ 1 on bounded domains [4], we refer to [24] where a minimization problem related to (1) is solved for Ω = IR n and to [27] for a similar problem in a bounded domain.
Besides the existence results, we believe that the interest of this paper are the techniques involved in the proofs: they generalize several well-known arguments of classical critical point theory.Since the celebrated mountainpass Lemma [1] (MPL in the sequel) much progress has been made to prove existence results for elliptic problems: we will not concern ourselves in the difficult task of giving complete references, but let us underline some possible modifications of the assumptions of the classical MPL.The first assumption of the MPL is the smoothness of the energy functional associated to the problem: as already mentioned, the functional J in ( 2) is nonsmooth and we need to apply the generalized critical point theory of [14,15].
The second basic assumption of the MPL is the Palais-Smale condition: this is a compactness condition which is guaranteed, in particular, for problems having subcritical growth and on bounded domains Ω.For certain problems having critical growth one may still prove that the Palais-Smale condition holds at certain energy levels, see [5,8,12,19]; for the functional J in (2), even in the nontrivial energy range determined by Lemma 2 below, the Palais-Smale condition (in the sense of [14,15]) does not hold because J may have negative critical levels, see Section 6.We overcome this problem by means of Lemma 3 below which is, somehow, the heart of our proofs: it states that the compactness is recovered if the loss of energy between the Palais-Smale sequence and its weak limit is less than a suitable threshold.This fact seems to be related to the representation of the D 1,2 0 -weakly convergent sequences of the concentration-compactness principle [22] and to the representation result for Palais-Smale sequences in [25]: such representation seems difficult to obtain in our case because the functional is nonsmooth.
Finally, to prove the existence of a positive solution of (1) in an unbounded domain Ω, we first solve (1) in a sequence of bounded subdomains Ω k of Ω and then prove that the corresponding sequence of solutions {u k } converges in some sense to a solution u of (1) in Ω; this method seems to have been first applied in [16,17], while for more general semilinear problems at critical growth we refer to [23].The main difficulty of this procedure is to prove that the solution u found as (weak) limit of the sequence {u k } is not the trivial one; one usually reasons by contradiction and proves that if u ≡ 0, then u k → 0 in the D 1,2  0 -norm topology: the contradiction is achieved if one proves a uniform lower estimate for the norm of u k , see e.g.[23].Our approach is slightly different as we do not obtain such estimate but we use again Lemma 3: on the other hand, a direct approach as in [5] seems not possible since it also makes use of a representation result for the Palais-Smale sequences.

Existence of a positive solution
Let us first list the assumptions on the functions which appear in (1): since some of them look quite technical, we refer to Section 6 for several examples.
We assume that the coefficients a ij (i, j = 1, ..., n) satisfy and, for simplicity, we assume that the functions a ij (x, s) are even with respect to s.Moreover, we require the ellipticity condition we also need an estimate for the growth of ellipticity Finally, as in [4], we require that (1) "converges" to a semilinear equation as u → +∞: ∀i, j = 1, . . ., n and uniformly w.r.t.x ∈ Ω . ( We require that the function g be subcritical: for a.e.x ∈ Ω and ∀s ∈ IR . ( Next, we need some local growth conditions on the function G(x, s) = s 0 g(x, t)dt: we assume that there exists a nonempty open set Ω 0 ⊂ Ω such that Without loss of generality we may assume that the origin 0 ∈ Ω 0 .Moreover, we require that where ν is the ellipticity constant in (4) and S is the best Sobolev constant of the embedding D 1,2 0 ⊂ L 2 * , see [26].Note that the above assumptions do not exclude that In fact, this is the interesting case because (1) admits u ≡ 0 as a solution and one needs to prove the existence of another (nontrivial) solution.
Finally, we assume that the function p is measurable and We first prove an existence result for bounded domains: Theorem 1. Assume that Ω is a bounded open subset of IR n and assume ( 3)-( 12); then, there exists at least a positive function u ∈ D 1,2 0 (Ω) solving (1) in the weak sense.
The proof of Theorem 1 follows the same steps as in [4].Nevertheless, the presence of the function p implies some changes both in the "nontrivial energy range" (see Lemma 2 below) and in the estimates involving the "Sobolev concentrating functions" (see (21)
To prove Theorem 2, we introduce a sequence of smooth bounded domains Ω k such that ∪ k∈IN Ω k = Ω: by Theorem 1, for all k, (1) admits a positive solution As in [23] the solution of (1) in Ω is obtained as weak limit of the sequence {u k }: a careful analysis is needed to prove that the weak limit is nontrivial, see Section 5.

The case of a bounded domain
In this section we assume that Ω is bounded and we prove Theorem 1; throughout this section we assume (3)-( 12): in fact, some of the lemmas below do not need the whole set of assumptions.
First observe that if S denotes the best Sobolev constant of the embedding D 1,2  0 ⊂ L 2 * , then by (12) and Sobolev inequality we get and therefore J (u)[u] is well defined for all u ∈ D 1,2 0 (Ω) and can be written in integral form.
Define the cone of positive functions and the functional By the same procedure used in [13], one can prove that if then u is a weak positive solution of (1).Therefore, without loss of generality we assume that g(x, s) = 0 ∀s ≤ 0 , for a.e.x ∈ Ω, and, to prove Theorem 1, we seek critical points of the functional For simplicity, we have dropped the index + on J.By reasoning as in Lemmas 3.3 and 3.4 in [4] one can prove Lemma 1.Let {u m } ⊂ D 1,2 0 (Ω) be a Palais-Smale sequence for J; then, there exists u ∈ D 1,2  0 (Ω) such that u m u in D 1,2 0 (up to a subsequence) and u solves (1) in the weak sense.
Let us recall that, using Theorem 2.1 in [6], one can also prove (see [4,9]) that for a.e.x ∈ Ω .(16) We now determine the nontrivial energy range of the functional J, namely, the energy levels for which the Palais-Smale sequences do not converge weakly to the trivial function: as we already pointed out, it seems not possible to determine a compactness range as in [8].The energy range of the next Lemma was first determined in [12], see also [11,19].Lemma 2. Let C be as in (15), let {u m } ⊂ C be a Palais-Smale sequence for J at level and assume that u m u.Then u ≡ 0.
Proof.By contradiction, assume u ≡ 0.Then, by (8), By reasoning as in Lemma 3.5 in [4] we arrive at and ( 19) Hence, by (17) we get and by (14) we have If u m → 0 we contradict c > 0. Therefore, and, by (19), (20) we get . Next we prove the existence of a Palais-Smale sequence in C whose level is in the nontrivial energy range of the functional J.We follow the idea of [8] and consider the family of functions which solve the equation 2 * = S n/2 .Let η be a positive smooth cut-off function with compact support in B ρ ⊂ Ω 0 and let u ε = ηu * ε .Since m > 0 in (12), for all ε > 0 there exists t ε > 0 such that J(tu ε ) < 0 for all t ≥ t ε ; define the class of paths (to simplify notations we do not highlight the dependence of Γ on ε) and the minimax value

J(γ(t)) . (22)
We obtain a Palais-Smale sequence for J at level α by applying the MPL in the nonsmooth version [15], see also Theorem 2.1 in [2]: indeed, in a standard way one can verify that the functional J has such geometrical structure; hence, α > 0.Moreover, as J(|u|) ≤ J(u) for all u ∈ D 1,2 0 (Ω), in (22) we can consider instead of the class Γ, the class of paths at lower levels, namely therefore, by the nonsmooth deformation Lemma [14] we may assume that the Palais-Smale sequence is in C.
To prove that α < S n/2 n M (2−n)/2 we claim that for small enough ε we have To this end, for all ε > 0 let t ε > 0 be such that We write J(t ε u ε ) as and we estimate all its terms.Since {u ε } is uniformly bounded in D 1,2 0 , if t ε → 0 then J(t ε u ε ) → 0 and we are done; moreover, t ε → +∞ implies J(t ε u ε ) → −∞ which is impossible: so it remains to consider the case where the sequence {t ε } is upper and lower bounded by two positive constants.Recall the following estimates (see [8,20]) as ε → 0 Thus, by reasoning as in [20,21], one obtains (as ε → 0) As in [4] we infer that there exists a function τ = τ (ε) such that lim ε→0 τ (ε) = +∞ and such that for ε small enough we have Finally, note that ( 5) and (7) imply Therefore, as t ε u ε ∈ C we have Hence, by (24) and ( 25) we obtain which proves (23) for small enough ε.We have thus obtained a Palais-Smale sequence (in C) for J at level α ∈ 0, S n/2 n M (2−n)/2 .Its weak limit is nonnegative because J + (u)[ϕ] = 0 for all ϕ ∈ C ∞ c (Ω) and nontrivial by Lemma 2.Moreover, it solves (1) by Lemma 1.

Compactness of the Palais-Smale sequences
In this section we prove the following crucial result: 0 (Ω).Proof.We first introduce some notations.For all s > 0, define the function By ε(x) we denote a generic function such that In the sequel, the same symbol ε(x) may denote different functions and we will often use the relation ε(x) + ε(x) = ε(x).For all s > 0 define Finally, some of the limits below are in fact limsup or liminf but we will not specify this fact since all our results hold up to a subsequence.Before starting we recall that u ∈ C, that we may assume {u m } ⊂ C and that u m u.
STEP 2: we prove that for all s > 0 we have We claim that (31) to this end, it is equivalent to prove that (32) = 0 for all m and we obtain then, by (29) and by Lebesgue's Theorem we infer This, together with ( 16) and Egorov's Theorem yields (32) and, consequently, (31).Finally, by ( 4) and (31) we deduce which, combined with (28), yields (30).
STEP 3: we prove that (33) By (7) (and the boundedness of D i u m 2 ) we get (33) then follows taking into account (30) and the arbitrariness of s.
STEP 6: conclusion.By (34) and (36) we clearly have ) therefore, by (14) we obtain Therefore, Lemma 3 follows if we can exclude the second alternative; by Theorem 1 in [7] we get by ( 35) and (36 then, by (27) and by letting m → ∞ we infer which proves that the second alternative of (38) does not hold and completes the proof of Lemma 3.

The case of an unbounded domain
In this section we prove Theorem 2: we assume (3)-( 13) and that Ω is unbounded.
We introduce a sequence of open bounded smooth domains for all k let α k be the minimax value defined in (22) relative to Ω k : clearly, the sequence {α k } is nonincreasing and ∀k .(39) By Theorem 1, for all k, (1) admits a positive solution u k ∈ D 1,2  0 (Ω k ): we denote by J k the functional corresponding to (1) in Ω k ; therefore By extending u k to be 0 in Ω \ Ω k , the sequence {u k } may be regarded as a subset of D 1,2 0 (Ω).We first prove Lemma 4.There exists u ∈ D 1,2 0 (Ω) such that u k u in D 1,2 0 (Ω), up to a subsequence; moreover, u is a nonnegative solution of (1) on Ω.
Proof.By (39) and Lemma 3 we obtain on the other hand, (8) implies (for all ε > 0) that To prove that the sequence {u k } is bounded we argue by contradiction and assume the converse; then, from ( 12), ( 41) and (42) we easily infer that (43) From ( 41) and (43) we get By ( 40) and ( 6) we infer that for all k ∈ IN we have which, by definition of γ, contradicts (44) if u k → ∞.Therefore, up to a subsequence, we have u k u in D 1,2 0 (Ω).By the extension of the result of [6] to unbounded domains (which has already been used in [13]) one obtains that (16) (with u m replaced by u k ) still holds; to prove that u solves (1), one can argue as for (2.3.2) in [9]: indeed, as noted in [4], one can still deduce (2.3.5) from (2.3.4) in [9] by using the strong convergence of ϕp exp{−Mu k } in L 2 * loc (Ω).The fact that u(x) ≥ 0 for a.e.x ∈ Ω follows from the pointwise convergence lim k→∞ u k (x) → u(x) for a.e.x ∈ Ω.
The main problem is that the function u found in Lemma 4 may be the trivial one; hence, to complete the proof of Theorem 2 we need to prove the following Lemma 5.The function u found in Lemma 4 is a positive solution of (1) in Ω.
Proof.Let ε(x), η s and the limits have the same meaning as in the proof of Lemma 3. Let {u k } denote the sequence of (positive) solutions of (1) on Ω k and denote by {u k m } the Palais-Smale sequence used in Section 3 to obtain u k .By Lemma 4, u k u and u is a nonnegative solution of (1) on Ω; by contradiction, assume that u ≡ 0. For all u ∈ D 1,2 0 , by ( 4) and (10) we have consider the function f (x) = c 1 x 2 − c 2 x 2 * ; clearly, there exists ρ > 0 where f attains its maximum on IR + .Define ᾱ := f (ρ), then hence, (45) follows by (22).
STEP 2: we prove that u k m → u k in D 1,2 0 (Ω) for all k, except for at most a finite number.
As we are assuming that u k 0, by Theorem 2.2.7 in [10], by (6) and by (51) we obtain Hence, by ( 4) and (39) we obtain for sufficiently large k (say for k ≥ k); then, by Lemma 3 we obtain Let ω be any open bounded subset of Ω and denote by R ω the supremum of the positive R for which Ω ∩ B R ⊂ ω (here B R = {x ∈ IR n ; |x| ≤ R}); in (47) take ϕ ∈ C ∞ c (Ω) such that ϕ ≡ 1 on ω and ϕ ≥ 0 in Ω: by ( 4) and ( 5) the two terms in (47) are positive, therefore, for all such domain ω we have Then, by ( 7) and ( 13) we obtain ( 48) Therefore, by the arbitrariness of s and ω, if we consider (40) we obtain (46).STEP 4: conclusion.
By step 2 we have J k (u k ) = α k for all k except for at most a finite number; then, by ( 41) and (45) we have up to a subsequence.Taking into account (46) and the first of (48), and by reasoning as in step 6 in the proof of Lemma 3 (with u ≡ 0), we obtain the following alternative either u k → 0 or lim k→∞ u k 2 ≥ S n/2 M (n−2)/2 ; in both cases we contradict (49): the contradiction is achieved and therefore u ≡ 0.
As already mentioned in the introduction, it seems impossible to recover compactness even in the nontrivial energy range determined in Lemma 2: in the semilinear case, one of the basic tools to obtain such compactness is the fact that the critical levels of the functional are positive, see e.g.Lemma 2.3(i) in [11] and (I 6 ) in [12].To this end, when a ij (x, s) ≡ δ ij , one may assume that (see e.g.[21]) In the quasilinear case the critical levels may be negative: assume that u is a critical point for J, then J (u)[u] = 0 and therefore we can write either In the first case, even if we assume (50), we cannot conclude that J(u) ≥ 0 because of (5).In the second case, even if we assume (6), we cannot conclude that J(u) ≥ 0 since (8) is incompatible with the assumption that g(x, s)s ≥ 2 * G(x, s) for all x, s.
To finish, let us mention some possible alternative assumptions under which Theorems 1 and 2 remain true.
• The assumption m > 0 in (12) is needed to prove the boundedness of the Palais-Smale sequences for the functional relative to (1) on a bounded domain and to obtain a mountain-pass geometry for the functional J, see (22): one could instead require more stringent conditions on the lower order term g, see assumption (H 3 ) in [23].
• In the case n = 4, ( 9) can be replaced by different conditions on the behaviour of G on Ω 0 , see [4].
• The flatness assumption (12) for x near 0 may be modified according to the behaviour of G at +∞, see [20].In the semilinear case, in bounded star-shaped domains one has non-existence results if p is not sufficiently flat at 0, see [18].If the supremum M of p is not attained on an unbounded domain, one needs additional assumptions on p to obtain existence results, see [16].For bounded domains, the case where p attains its maximum at a boundary point has been studied in [19].Note also that by reasoning as in [12] one could extend our results to the case where the function p(x) in ( 1) is replaced by a suitable function p(x, u).

Lemma 3 .
Denote by {u m } the Palais-Smale sequence used in Section 3 to obtain u.If