Let R be a prime ring of char R≠2, d a nonzero derivation of R and ρ a nonzero right ideal of R such that [[d(x),x]n,[y,d(y)]m]t=0 for all x,y∈ρ, where n≥0, m≥0, t≥1 are fixed integers. If [ρ,ρ]ρ≠0, then d(ρ)ρ=0.

1. Introduction

Throughout this paper, unless specifically stated, R always denotes a prime ring with center Z(R) and extended centroid C, Q the Martindale quotients ring. Let n be a positive integer. For given a,b∈R, let [a,b]0=a and let [a,b]1 be the usual commutator ab-ba, and inductively for n>1, [a,b]n=[[a,b]n-1,b]. By d we mean a nonzero derivation in R.

A well-known result proven by Posner [1] states that if [[d(x),x],y]=0 for all x,y∈R, then R is commutative. In [2], Lanski generalized this result of Posner to the Lie ideal. Lanski proved that if U is a noncommutative Lie ideal of R such that [[d(x),x],y]=0 for all x∈U,y∈R, then either R is commutative or char R=2 and R satisfies S4, the standard identity in four variables. Bell and Martindale III [3] studied this identity for a semiprime ring R. They proved that if R is a semiprime ring and [[d(x),x],y]=0 for all x in a non-zero left ideal of R and y∈R, then R contains a non-zero central ideal. Clearly, this result says that if R is a prime ring, then R must be commutative.

Several authors have studied this kind of Engel type identities with derivation in different ways. In [4], Herstein proved that if char R≠2 and [d(x),d(y)]=0 for all x,y∈R, then R is commutative. In [5], Filippis showed that if R is of characteristic different from 2 and ρ a non-zero right ideal of R such that [ρ,ρ]ρ≠0 and [[d(x),x],[d(y),y]]=0 for all x,y∈ρ, then d(ρ)ρ=0.

In continuation of these previous results, it is natural to consider the situation when [[d(x),x]n,[y,d(y)]m]t=0 for all x,y∈ρ, n,m≥0,t≥1 are fixed integers. We have studied this identity in the present paper.

It is well known that any derivation of a prime ring R can be uniquely extended to a derivation of Q, and so any derivation of R can be defined on the whole of Q. Moreover Q is a prime ring as well as R and the extended centroid C of R coincides with the center of Q. We refer to [6, 7] for more details.

Denote by Q*CC{X,Y} the free product of the C-algebra Q and C{X,Y}, the free C-algebra in noncommuting indeterminates X,Y.

2. The Case: <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M77"><mml:mrow><mml:mi>R</mml:mi></mml:mrow></mml:math></inline-formula> Prime Ring

We need the following lemma.

Lemma 2.1.

Let ρ be a non-zero right ideal of R and d a derivation of R. Then the following conditions are equivalent: (i) d is an inner derivation induced by some b∈Q such that bρ=0; (ii) d(ρ)ρ=0 (for its proof refer to [8, Lemma]).

We mention an important result which will be used quite frequently as follows.

Theorem 2.2 (see Kharchenko [<xref ref-type="bibr" rid="B11">9</xref>]).

Let R be a prime ring, d a derivation on R and I a non-zero ideal of R. If I satisfies the differential identity f(r1,r2,…,rn,d(r1),d(r2),…,d(rn))=0foranyr1,r2,…,rn∈I, then either (i) I satisfies the generalized polynomial identity
f(r1,r2,…,rn,x1,x2,…,xn)=0,
or (ii) d is Q-inner, that is, for some q∈Q,d(x)=[q,x] and I satisfies the generalized polynomial identity
f(r1,r2,…,rn,[q,r1],[q,r2],…,[q,rn])=0.

Theorem 2.3.

Let R be a prime ring of char R≠2 and d a derivation of R such that [[d(x),x]n,[[y,d(y)]m]t=0 for all x,y∈R, where n≥0,m≥0,t≥1 are fixed integers. Then R is commutative or d=0.

Proof.

Let R be noncommutative. If d is not Q-inner, then by Kharchenko's Theorem [9]
g(x,y,u,v)=[[u,x]n,[y,v]m]t=0,
for all x,y,u,v∈R. This is a polynomial identity and hence there exists a field F such that R⊆Mk(F) with k>1, and R and Mk(F) satisfy the same polynomial identity [10,Lemma 1]. But by choosing u=e12,x=e11,v=e11 and y=e21, we get
0=[[u,x]n,[y,v]m]t=(-1)tn(e11+(-)te22),
which is a contradiction.

Now, let d be Q-inner derivation, say d=ad(a) for some a∈Q, that is, d(x)=[a,x] for all x∈R, then we have
[[a,x]n+1,[y,[a,y]]m]t=0,
for all x,y∈R. Since d≠0, a∉C and hence R satisfies a nontrivial generalized polynomial identity (GPI). By [11], it follows that RC is a primitive ring with H=Soc(RC)≠0, and eHe is finite dimensional over C for any minimal idempotent e∈RC. Moreover we may assume that H is noncommutative; otherwise, R must be commutative which is a contradiction.

Notice that H satisfies [[a,x]n+1,[y,[a,y]]m]t=0 (see [10, Proof of Theorem 1]). For any idempotent e∈H and x∈H, we have
0=[[a,e]n+1,[ex(1-e),[a,ex(1-e)]]m]t.
Right multiplying by e, we get
0=[[a,e]n+1,[ex(1-e),[a,ex(1-e)]]m]te=[[a,e]n+1,[ex(1-e),[a,ex(1-e)]]m]t-1·{[a,e]n+1([ex(1-e),[a,ex(1-e)]]m)e-([ex(1-e),[a,ex(1-e)]]m)[a,e]n+1e}=[[a,e]n+1,[ex(1-e),[a,ex(1-e)]]m]t-1·{[a,e]n+1(∑j=0m(-1)j(mj)[a,ex(1-e)]jex(1-e)[a,ex(1-e)]m-j)e-(∑j=0m(-1)j(mj)[a,ex(1-e)]jex(1-e)[a,ex(1-e)]m-j)[a,e]n+1e}=[[a,e]n+1,[ex(1-e),[a,ex(1-e)]]m]t-1·{0-(∑j=0m(-1)j(mj)(-ex(1-e)a)jex(1-e)(aex(1-e))m-j)ae}=-[[a,e]n+1,[ex(1-e),[a,ex(1-e)]]m]t-1(∑j=0m(mj)(ex(1-e)a)m+1)e=-2m[[a,e]n+1,[ex(1-e),[a,ex(1-e)]]m]t-1(ex(1-e)a)m+1e=(-)t2mt(ex(1-e)a)(m+1)te.

This implies that 0=(-)t2mt((1-e)aex)(m+1)t+1. Since char R≠2, ((1-e)aex)(m+1)t+1=0. By Levitzki's lemma [12, Lemma 1.1], (1-e)aex=0 for all x∈H. Since H is prime ring, (1-e)ae=0, that is, eae=ae for any idempotent e∈H. Now replacing e with 1-e, we get that ea(1-e)=0, that is, eae=ea. Therefore for any idempotent e∈H, we have [a,e]=0. So a commutes with all idempotents in H. Since H is a simple ring, either H is generated by its idempotents or H does not contain any nontrivial idempotents. The first case gives a∈C contradicting d≠0. In the last case, H is a finite dimensional division algebra over C. This implies that H=RC=Q and a∈H. By [10,Lemma 2], there exists a field F such that H⊆Mk(F) and Mk(F) satisfies [[a,x]n+1,[y,[a,y]]m]t. Then by the same argument as earlier, a commutes with all idempotents in Mk(F), again giving the contradiction a∈C, that is, d=0. This completes the proof of the theorem.

Theorem 2.4.

Let R be a prime ring of char R≠2, d a non-zero derivation of R and ρ a non-zero right ideal of R such that [[d(x),x]n,[y,d(y)]m]t=0 for all x,y∈ρ, where n≥0,m≥0,t≥1 are fixed integers. If [ρ,ρ]ρ≠0, then d(ρ)ρ=0.

We begin the proof by proving the following lemma.

Lemma 2.5.

If d(ρ)ρ≠0 and [[d(x),x]n,[y,d(y)]m]t=0 for all x,y∈ρ,m,n≥0,t≥1 are fixed integers, then R satisfies nontrivial generalized polynomial identity (GPI).

Proof.

Suppose on the contrary that R does not satisfy any nontrivial GPI. We may assume that R is noncommutative; otherwise, R satisfies trivially a nontrivial GPI. We consider two cases.Case 1.

Suppose that d is Q-inner derivation induced by an element a∈Q. Then for any x∈ρ,[[a,xX]n+1,[xY,[a,xY]]m]t
is a GPI for R, so it is the zero element in Q*CC{X,Y}. Expanding this, we get
([a,xX]n+1∑j=0m(-1)j(mj)[a,xY]jxY[a,xY]m-j-∑j=0m(-1)j(mj)[a,xY]jxY[a,xY]m-j[a,xX]n+1)A(X,Y)=0,
where A(X,Y)=[[a,xX]n+1,[xY,[a,xY]]m]t-1. If ax and x are linearly C-independent for some x∈ρ, then
((axX)n+1∑j=0m(-1)j(mj)[a,xY]jxY[a,xY]m-j-∑j=0m(-1)j(mj)(axY)jxY[a,xY]m-j[a,xX]n+1)A(X,Y)=0.
Again, since ax and x are linearly C-independent, above relation implies that
(-xY[a,xY]m[a,xX]n+1)A(X,Y)=0,
and so
(-xY(axY)m(axX)n+1)A(X,Y)=0.
Repeating the same process yields
(-xY(axY)m(axX)n+1)t=0
in Q*CC{X,Y}. This implies that ax=0, a contradiction. Thus for any x∈ρ, ax and x are C-dependent. Then (a-α)ρ=0 for some α∈C. Replacing a with a-α, we may assume that aρ=0. Then by Lemma 2.1, d(ρ)ρ=0, contradiction.

Case 2.

Suppose that d is not Q-inner derivation. If for all x∈ρ, d(x)∈xC, then [d(x),x]=0 which implies that R is commutative (see [13]). Therefore there exists x∈ρ such that d(x)∉xC, that is, x and d(x) are linearly C-independent.

By our assumption, we have that R satisfies

[[d(xX),xX]n,[xY,d(xY)]m]t=0.
By Kharchenko's Theorem [9],
[[d(x)X+xr1,xX]n,[xY,d(x)Y+xr2]m]t=0,
for all X,Y,r1,r2∈R. In particular for r1=r2=0,
[[d(x)X,xX]n,[xY,d(x)Y]m]t=0,
which is a nontrivial GPI for R, because x and d(x) are linearly C-independent, a contradiction.

We are now ready to prove our main theorem.

Proof of Theorem <xref ref-type="statement" rid="thm1.3">2.4</xref>.

Suppose that d(ρ)ρ≠0, then we derive a contradiction. By Lemma 2.5, R is a prime GPI ring, so is also Q by [14]. Since Q is centrally closed over C, it follows from [11] that Q is a primitive ring with H=Soc(Q)≠0.

By our assumption and by [7], we may assume that
[[d(x),x]n,[y,d(y)]m]t=0
is satisfied by ρQ and hence by ρH. Let e=e2∈ρH and y∈H. Then replacing x with e and y with ey(1-e) in (2.17), then right multiplying it by e, we obtain that
0=[[d(e),e]n,[ey(1-e),d(ey(1-e))]m]te=[[d(e),e]n,[ey(1-e),d(ey(1-e))]m]t-1·{[d(e),e]n∑j=0m(-1)j(mj)d(ey(1-e))jey(1-e)d(ey(1-e))m-je-∑j=0m(-1)j(mj)d(ey(1-e))jey(1-e)d(ey(1-e))m-j[d(e),e]ne}.

Now we have the fact that for any idempotent e, d(y(1-e))e=-y(1-e)d(e), ed(e)e=0 and so
0=[[d(e),e]n,[ey(1-e),d(ey(1-e))]m]t-1·{0-∑j=0m(-1)j(mj)e(-y(1-e)d(e))jy(1-e)d(ey(1-e))m-jd(e)e}.
Now since for any idempotent e and for any y∈R, (1-e)d(ey)=(1-e)d(e)y, above relation gives
0=[[d(e),e]n,[ey(1-e),d(ey(1-e))]m]t-1·{-e∑j=0m(mj)(y(1-e)d(e))jy(1-e)(d(e)y(1-e))m-jd(e)e}=[[d(e),e]n,[ey(1-e),d(ey(1-e))]m]t-1{-e∑j=0m(mj)(y(1-e)d(e))m+1e}=[[d(e),e]n,[ey(1-e),d(ey(1-e))]m]t-1{-2me(y(1-e)d(e))m+1e}={-2me(y(1-e)d(e))m+1}te.
This implies that 0=(-1)t2mt((1-e)d(e)ey)(m+1)t+1 for all y∈H. Since char R≠2, we have by Levitzki's lemma [12,Lemma 1.1] that (1-e)d(e)ey=0 for all y∈H. By primeness of H, (1-e)d(e)e=0. By [15,Lemma 1], since H is a regular ring, for each r∈ρH, there exists an idempotent e∈ρH such that r=er and e∈rH. Hence (1-e)d(e)e=0 gives (1-e)d(e)=(1-e)d(e2)=(1-e)d(e)e=0 and so d(e)=ed(e)∈eH⊆ρH and d(r)=d(er)=d(e)er+ed(er)∈ρH. Hence for each r∈ρH, d(r)∈ρH. Thus d(ρH)⊆ρH. Set J=ρH. Then J¯=J/(J∩lH(J)), a prime C-algebra with the derivation d¯ such that d¯(x¯)=d(x)¯, for all x∈J. By assumption, we have that
[[d¯(x¯),x¯]n,[y¯,d¯(y¯)]m]t=0,
for all x¯,y¯∈J¯. By Theorem 2.3, we have either d¯=0 or ρH¯ is commutative. Therefore we have that either d(ρH)ρH=0 or [ρH,ρH]ρH=0. Now d(ρH)ρH=0 implies that 0=d(ρρH)ρH=d(ρ)ρHρH and so d(ρ)ρ=0. [ρH,ρH]ρH=0 implies that 0=[ρρH,ρH]ρH=[ρ,ρH]ρHρH and so [ρ,ρH]ρ=0, then 0=[ρ,ρρH]ρ=[ρ,ρ]ρHρ implying that [ρ,ρ]ρ=0. Thus in all the cases we have contradiction. This completes the proof of the theorem.

3. The Case: <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M323"><mml:mrow><mml:mi>R</mml:mi></mml:mrow></mml:math></inline-formula> Semiprime Ring

In this section we extend Theorem 2.3 to the semiprime case. Let R be a semiprime ring and U be its right Utumi quotient ring. It is well known that any derivation of a semiprime ring R can be uniquely extended to a derivation of its right Utumi quotient ring U and so any derivation of R can be defined on the whole of U [7,Lemma 2].

By the standard theory of orthogonal completions for semiprime rings, we have the following lemma.

Lemma 3.1 (see [<xref ref-type="bibr" rid="B1">16</xref>, Lemma <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M331"><mml:mrow><mml:mn>1</mml:mn></mml:mrow></mml:math></inline-formula> and Theorem <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M332"><mml:mrow><mml:mn>1</mml:mn></mml:mrow></mml:math></inline-formula>] or [<xref ref-type="bibr" rid="B14">7</xref>,pages 31-32]).

Let R be a 2-torsion free semiprime ring and P a maximal ideal of C. Then PU is a prime ideal of U invariant under all derivations of U. Moreover, ⋂{PU∣PisamaximalidealofCwithU/PU2-torsionfree}=0.

Theorem 3.2.

Let R be a 2-torsion free semiprime ring and d a non-zero derivation of R such that [[d(x),x]n,[y,d(y)]m]t=0 for all x,y∈R, n,m≥0,t≥1 fixed are integers. Then d maps R into its center.

Proof.

Since any derivation d can be uniquely extended to a derivation in U, and R and U satisfy the same differential identities [7, Theorem 3], we have
[[d(x),x]n,[y,d(y)]m]t=0,
for all x,y∈U. Let P be any maximal ideal of C such that U/PU is 2-torsion free. Then by Lemma 3.1, PU is a prime ideal of U invariant under d. Set U¯=U/PU. Then derivation d canonically induces a derivation d¯ on U¯ defined by d¯(x¯)=d(x)¯ for all x∈U. Therefore,
[[d¯(x¯),x¯]n,[y¯,d¯(y¯)]m]t=0,
for all x¯,y¯∈U¯. By Theorem 2.3, either d¯=0 or [U¯,U¯]=0, that is, d(U)⊆PU or [U,U]⊆PU. In any case d(U)[U,U]⊆PU for any maximal ideal P of C. By Lemma 3.1, ⋂{PU∣PisamaximalidealofCwithU/PU2-torsionfree}=0. Thus d(U)[U,U]=0. Without loss of generality, we have d(R)[R,R]=0. This implies that
0=d(R2)[R,R]=d(R)R[R,R]+Rd(R)[R,R]=d(R)R[R,R].
Therefore [R,d(R)]R[R,d(R)]=0. By semiprimeness of R, we have [R,d(R)]=0, that is, d(R)⊆Z(R). This completes the proof of the theorem.

PosnerE. C.Derivations in prime ringsLanskiC.Differential identities, Lie ideals, and Posner's theoremsBellH. E.MartindaleW. S.IIICentralizing mappings of semiprime ringsHersteinI. N.A note on derivationsDe FilippisV.On derivations and commutativity in prime ringsBeidarK. I.MartindaleW. S.IIIMikhalevA. V.LeeT. K.Semiprime rings with differential identitiesBrešarM.One-sided ideals and derivations of prime ringsKharchenkoV. K.Differential identities of prime ringsLanskiC.An Engel condition with derivationMartindaleW. S.IIIPrime rings satisfying a generalized polynomial identityHersteinI. N.BellH. E.DengQ.On derivations and commutativity in semiprime ringsChuangC.-L.GPI's having coefficients in Utumi quotient ringsFaithC.UtumiY.On a new proof of Litoff's theoremBeidarK. I.Rings of quotients of semiprime rings