COHYPONORMAL OPERATORS WITH THE SINGLE VALUED EXTENSION PROPERTY

It is proved that in order to find a nontrivial hyperinvariant subspace for a cohyponormal operator it suffices to make the further assumption that the operator have the single-valued extension property.

Let T be a cohyponormal operator on the complex Hi|bert space X.This means that T* is hyponormal, so lTxll < lT*xll for each vector x.The aim of this paper is to reduce the problem of existence of a nontrivial invariant (or hyperinvariant) subspace for T to the case where T has the single-valued extension property and is the compact perturbation of a normal operator (see 3).In {}2 we record some consequences of a recent theorem of Putinar [1] that every hyponormal operator is "subscalar".The last section gives some further invariant subspace results for this reduced case.
We mention a recent paper of S. Brown [2] in which the author shows that a hyponormal operator having spectrum with nonempty interior has invariant (but not necessarily hyperinvariant) subspaces.2. HYPONORMAL OPERATORS.
THEOREM 1.Let T be hyponormal on X.Then T has an extension S on a Hilbert space Y containing X such that S is a generalized scalar operator of order 2. In particular, S in decomposab|e [3, p. 67].
Every decomposable operator has property (B) [4], introduced by Bishop [5]; T has property (B) if, for each sequence of analytic functions fn:D X such that (-T)fn(}, 0 uniformly on compact sets, fn 0 uniformly on compact sets.Since it is clear that restrictions inherit (B), the following is immediate.COROLLARY 1.Every hyponormal operator has property (B).If T is hyponormal and has a nonzero invariant subspace M such that o(TIM) o(T) is proper, then T has a nontrivial hyperinvariant subspace.
PROOF.It follows from Corollary that the spectral manifold N XT(O(TIM)) is closed, hyperinvariant for T, and contains M. Since o(TIM) o(T), N is nontrivial.
Let T be cohyponormal on X.Then T is decomposable iff it has property ().
PROOF.If T has property (B), then it is decomposable by Corollary and [6].
The converse was noted in I.
REMARK I.
The left shift on 2(N) is cohyponormal but it does not have property (B)since it is not decomposable.On the other hand, there do exist nonnorma| decomposable cohyponormal operators satisfying (B).Radjabalipour [7] has proved the existence of nonnormal decomposable subnormal operators; adjoints of such operators are cohyponormal and satisfy the conclusion of Theorem 2.
COROLLARY 3. Let T be a cohyponormal operator which is not a scalar multiple of the identity on X.
If T satisfies (), then it has a nontrivial hyperinvariant subspace.

PROOF.
If o(T) is the singleton {}, then (L-T)* is a quasinilpotent hyponormal opertor.Hence T O, contradiction.Now T is decomposable by Theorem 2, and since o(T) has at least two points, T has a nontrivial, hyperinvariant spectral manifold XT(F).Let T be cohyponormal and have property (B).If for each hyperinvariant subspace M the restriction TIM is cohyponormal, then T is strongly decomposabl e.
PROOF.Let M XT(F for F closed.Since TIM is cohyponormal by hypothesis and inherits (B), then TIM is decomposable by Theorem 2. This is just the definition that T is strongly decomposable [8].
REMARK 2. Since t,he subnormal operators mentioned in Remark in fact have spectral distributions, their adjoints are strongly decomposable cohyponormal operators but nonnormal (see [3, p. 65 and p. 81]).In general it is not known if (B)is preserved under the Dunford functional calculus, but it is in the case of cohyponormal operators (Corollary 5).We note that (B) is also preserved in this way for hyponormal operators as well.For let T be hyponormal on X and let f be analytic on some region containing o(T).If S is the scalar extension of Theorem I, then o(S)= (T) [I, Cor 2.6] so f(S) is defined and decomposable [3, p. 37] and f(T) f(S)IX.Thus f(T) satisfies (B).COROLLARY 5. Let T be cohyponormal and let f be analytic on o(T).Then f(T) satisfies (B) if T does.Conversely, if f is nonconstant on each component of its domain, then T satisfies (B) if f(T) does.
PROOF.If T has property (B), then it is decomposable by Theorem 2. Hence f(T) is also decomposable and so satisfies (B) [4].For the converse, note that (the conjugate of f) is analytic on o(T*).Thus if S is the scalar extension of T* (Theorem i), then (S) is decomposable and (T*) has property (B).By [6] f(T) is decomposable, so by [8] T is decomposable and thus satisfies (B).REMARK 3. Corollary 5 may fail if f is anywhere constant.If T is the left shift and f 1, then f(T) has property (8) but T does not.Property (6) implies the (strictly) weaker single-valued extension property (SVEP) which has had a role in the theory of spectral decomposition (see [3, p. 1]).
We shall see below that it may also be significant in the invariant subspace problem for hyponormal operators.(An example in [3, p. 25] shows SVEP does not imply (8).
Let T be cohyponomal satisfying SVEP.If S is the scalar extension of T*, then o(S) o(T*). PROOF.
Then >, S, and hence T*, is bounded below.
Since T* is hyponomal, it is semi-Fredholm with index at most zero.
As corollaries to the previous proof we have the following: COROLLARY 6.If T is cohyponromal and has SVEP, then T is biquasitriangular.PROOF.For }, in the essential resolvent set of T* the previous proof implies ind( T*) O.By [I0] T*, and so T, is biquasitriangular.
COROLLARY 7. Let T be hyponomal with scalar extension S. If o(S) o(T), then T has a nontrivial hyperinvariant subspace.
PROOF.Since o(S) is proper in o(T), the proof of Proposition shows that T* does not have the SVEP.
Hence T* has an eigenvector, so T has a nontrivial hyperi nvariant subspace.COROLLARY 8. Let T be a cohyponomal operator which is not a scalar multiple of the identity.
Then T has a nontrivial hyperinvariant subspace if T has property (8) or T does not have SVEP.
PROOF.If T satisfies (8), use Corollary 3. In the other case T has at least one eigenvector.
By Coro|lary 8 the hyperinvariant subspace problem for a h,vponomal operator is reduced to the case where its adjoint has SVEP but not property (8).This leads to the related question: if T is cohyponomal does SVEP imply (B)?An affimative answer to the last question would, of course, resolve the former one.On the other hand, if our interest is merely invariant subspaces, we have: THEOREM 3. If T is cohyponomal, then either T has a nontrivial invariant subspace or T is a compact perturbation of a nomal operator. PROOF.
By Corollary 8 we may suppose that T has SVEP.If T* has a cyclic vector, we may suppose it is rationally cyclic.By [11, Th. 4] T*T TT* is in the trace class.By Proposition and [12, Th. 11.11] T N + K where N is normal and K s compact.
Our results now indicate that to solve the hyperinvariant subspace problem for cohyponomal T we may make the following additional assumptions: (a) T is the compact perturbation of a nomal operator and (T*) (S) where S is the scalar extension of T*.