International Journal of Mathematics and Mathematical Sciences
Volume 2018 (2018), Article ID 4867091, 6 pages
https://doi.org/10.1155/2018/4867091

Research Article

Greedy Expansions with Prescribed Coefficients in Hilbert Spaces

Artur R. Valiullin, Albert R. Valiullin, and Vladimir V. Galatenko

Faculty of Mechanics and Mathematics, Lomonosov Moscow State University, Russia

Correspondence should be addressed to Albert R. Valiullin; albert.valiullin1997@gmail.com

Received 28 October 2018; Accepted 15 November 2018; Published 9 December 2018

Academic Editor: Brigitte Forster-Heinlein

Copyright © 2018 Artur R. Valiullin et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Greedy expansions with prescribed coefficients, which have been studied by V. N. Temlyakov in Banach spaces, are considered here in a narrower case of Hilbert spaces. We show that in this case the positive result on the convergence does not require monotonicity of coefficient sequence . Furthermore, we show that the condition sufficient for the convergence, namely, the inclusion , can not be relaxed at least in the power scale. At the same time, in finite-dimensional spaces, the condition can be replaced by convergence of to zero.

1. Introduction

Expansion in Fourier series [1] is a classical and comprehensively studied tool of theoretical and applied mathematics which takes an expanded function as an input and constructs a sequence of its expansion coefficients. Greedy expansions [2, 3], which are equivalent in the simplest case to Fourier series reordered by decreasing norms of terms and known in statistics and signal processing as Projection Pursuit Regression [4, 5] and Matching Pursuit [6], respectively, perform parallel computation of expansion coefficients and selection of expansion elements from a predefined dictionary. V. N. Temlyakov [3, 7] (see also [8]) proposed a type of a greedy expansion that performs only selection of expansion elements, while coefficients are prescribed in advance. The definition proposed by V. N. Temlyakov for the case of Banach spaces, in the case of Hilbert spaces, takes the following form.

Definition 1. Let be a Hilbert space over with a scalar product , be a symmetric unit-normed dictionary in (i.e., , all elements in have a unit norm, and if , then also belongs to ). In addition, let , , and be a sequence of positive numbers. We define inductively a sequence of remainders and a sequence of expanding elements . First, we set . Then, if has already been defined, we select as an (arbitrary) element which satisfies the condition

and set .
The series is called a greedy expansion of in the dictionary with the prescribed coefficients and the weakness parameter .

It immediately follows from the definition of a greedy expansion that , and hence the convergence of the expansion to an expanded element is equivalent to the convergence of remainders to zero as .

As a selection of an expanding element is potentially not unique, there may exist different realizations of a greedy expansion for a given dictionary , weakness parameter and sequence of coefficients . Furthermore, for greedy expansion may turn out to be nonrealizable due to the absence of an element which provides .

V. N. Temlyakov showed [3, heorem 2.1] that if a number of conditions hold which are equivalent in the case of Hilbert spaces to the divergence of the series and the convergence of the series , a greedy expansion with prescribed coefficients converges to an expanded element at least for a subsequence of indexes, i.e., . Later V. N. Temlyakov proved the standard convergence (i.e., ) under the additional condition of monotonicity of [7, heorem 4]. Yet, it remained unknown whether the condition and the monotonicity condition could be essentially relaxed without violating the guaranteed convergence to an expanded element.

2. Main Results

We start with a positive result which states that in Hilbert spaces the monotonicity is not required for the standard convergence. Namely, the following theorem holds.

Theorem 2. Let be a Hilbert space, be a symmetric unit-normed dictionary in , , be a sequence of positive numbers which satisfies the conditions

(i.e., ). Then for every element all realizations of its greedy expansion in the dictionary with the prescribed coefficients and the weakness parameter converge to .

It is clear that if the first condition on is violated, then there is no convergence to an expanded element for all with the norm exceeding the sum . The significance of the second condition on follows from the following theorem.

Theorem 3. There exist a Hilbert space , a symmetric unit-normed dictionary , an element and a sequence of positive numbers such that

but a greedy expansion of in the dictionary with the prescribed coefficients and the weakness parameter does not converge to .

As for the second condition of Theorem 2 a boundary in the power scale is , Theorem 3 in fact shows that this condition in Theorem 2 can not be relaxed at least in the power scale.

However, the question about a possibility of a more delicate relaxation of the condition remains open. This question can be stated as follows: is it true that for every sequence of positive numbers that converges to zero but does not belong to there exist a Hilbert space , a symmetric unit-normed dictionary and an expanded element such that at least one realization of greedy expansion of in with the prescribed coefficients (and, e.g., the weakness parameter ) does not converge to ?

We note that assertions similar to Theorems 2 and 3 have been announced by O. Rassudova in her conference talk [9], but the proofs have not been published. To the best of our knowledge, in her proof of an analogue of Theorem 3 O. Rassudova used a modification of the construction [10, heorem 3] which is based on analytical estimates and does not have a clear geometric interpretation. The construction presented in our work is geometrically demonstrative.

We also note that at least for for the natural class of monotonic coefficients in case of finite-dimensional Hilbert spaces the condition in Theorem 2 can be replaced by an essentially weaker condition . The proof of this fact is presented in section The case of finite-dimensional spaces.

3. Proof of Theorem 2

The theorem can be easily derived from the equality , which holds due to the aforementioned result by V. N. Temlyakov [3, heorem 2.1]. From the definition of the greedy expansion it immediately follows that

As coefficients are positive and the dictionary is symmetric, . Hence

and thus

The condition implies that

Due to the equality we have that

From two last assertions we obtain that for every there exists such that the following two conditions simultaneously hold:

Thus using estimate (6) we get that for all . But according to the definition of the limit it directly means that . The proof of Theorem 2 is complete.

We note that for monotonic coefficients and the weakness parameter the statement of Theorem 2, which is covered in this case by [7, heorem 4], can be also derived as a corollary of the same result by V. N. Temlyakov about the convergence for a subsequence of indexes [3, heorem 2.1] and the following lemma. We find this lemma to be interesting on its own.

Lemma 4. Let and monotonically converge to zero as . Then for every realization of greedy expansion with the prescribed coefficients the sequence of norms of its remainders converges.

We begin our proof of this lemma with an estimate of a possible increase of the remainder norms. Let denote the set of all indexes for which . If the set is finite, then starting from a certain index the sequence is monotonic and thus convergent. Hence it remains to consider the case of countably infinite .

For the sake of brevity we denote the scalar products by . It follows from the definition of the greedy expansion and the symmetric property of the dictionary that all are nonnegative.

The Pythagorean theorem implies that for all indexes the equality

holds. If , then , and thus

Consequently

At the same time

It means that if the remainder norm increased at the transition from to , then the increase of the norm square does not exceed , for the next expansion step the increase is impossible, and the joint increase of the square of remainder norm for two steps of the expansion does not exceed and hence does not exceed , where can be set to .

Having this estimate, let us complete the proof. We note that the series converges: it can be easily derived either from the Leibniz’s alternating series test or from the inequality

Let us fix an arbitrary positive and find an index such that and simultaneously . Next we find an index such that , where denotes the infimum of the remainder norms . Then for every we have that

Hence () and consequently . The proof of Lemma 4 is complete.

4. Proof of Theorem 3

Our proof of Theorem 3 includes the following blocks: construction of a dictionary with simultaneous construction of coefficients ; description of realization of greedy expansion; proof of the absence of convergence to the expanded element; obtaining the required estimate of . As a Hilbert space we take an arbitrary infinite-dimensional separable space, e.g., .

Figure 1 illustrates certain steps of the proof.

Figure 1: An illustration for certain steps of the proof of Theorem 3. All vectors except lie in the vertical plane; (as well as ) lies in the horizontal plane; represents any vector from the set . The spherical law of cosines is applied to angles formed by the vectors , , . The law of sines is applied to the triangle formed by vectors , , .

4.1. Description of the Construction

We first present the structure of the example; i.e., we describe the construction of dictionary elements and coefficients . As a part of this construction we also define the sequence of vectors (remainders) , including the expanded element .

Let be an arbitrary non-zero element of with , . We define dictionary elements and as arbitrary (unequal) unit vectors such that , and lie in one plane and the angle between and equals the angle between and and belongs to the interval .

Next, we set dictionary element to an arbitrary unit vector with the following two properties: its orthogonal projection on the plane lies on the line with directing vector , and the angle between this vector and also equals . We also find the coefficient such that for scalar products are equal or, equivalently, angles between and the vectors , , are equal. We denote this angle by . Note that : the formal justification of this fact can be based, e.g., on the equality , which directly follows from the spherical law of cosines.

Similarly, we set to an arbitrary unit vector with an orthogonal projection on the subspace lying on the line with directing vector and the angle between this vector and equal , and select the coefficient in such a way that for scalar products , , , are equal or, equivalently, angles between and the vectors , , , are equal. We denote this angle by . Again it is easy to see that , as due to the spherical law of cosines .

We continue the construction inductively. Namely, after constructing , , and we set to an arbitrary unit vector with an orthogonal projection on the subspace lying on the line with directing vector and angle between this vector and equal , and select the coefficient in such a way that for scalar products , , , , , are equal or, equivalently, angles between and the vectors , , , , , , are equal. We denote this angle by and note that as due to the spherical law of cosines, .

Let us justify formally that it is possible to find and with the required properties. Let denote an arbitrary unit vector orthogonal to the subspace (which also contains vector ), and denote the set of vectors . As we can take any of two unit vectors from the plane that have an angle with equal to . Let ; note that as the angle is acute. By construction for all the scalar product equals . For the sake of brevity we denote by . Hence for all , as , scalar products of and are the same and equal . Consequently, for an arbitrary positive and every we have equalities , . Thus, it remains to set to the solution of the linear equation , i.e., to .

4.2. Realization of Greedy Expansion

We note that a possible realization of greedy expansion in the dictionary

with the prescribed coefficients (and the weakness parameter ) is a realization in which is selected as an expanding element at the -th step, and hence -th remainder coincides with . Indeed, an angle between a vector and its orthogonal projection on a subspace does not exceed an angle between this vector and any non-zero vector from the subspace. Thus while for every and every we have an equality

and for the scalar product is positive, for we have an inequality

4.3. Absence of Convergence

In this subsection we show that the greedy expansion does not converge to or, equivalently, remainders do not converge to zero.

First we find the limit of . It exists as is a nondecreasing sequence with all values belonging to the interval . As noted above, for all the equality

holds. Consequently, if denotes the limit , then . It implies that .

Now we show that . Indeed,

Thus

As , and hence .

The law of sines gives the equalities

Consequently

Taking into consideration the convergence , we derive from this equality that

so the absence of convergence to the expanded element is proved.

4.4. Estimate of

Here we first show that . Applying again the law of sines, but this time for the other pairs of angles, we get equalities

Let us use equality (23):

Taking into account the monotonicity of , it implies the inequality

Consequently, as the series diverges, the series also diverges.

Then we show that . The equality (26) and monotonicity of imply that

It remains to establish the inequality : as , it directly gives the required upper estimate of . For this inequality holds because . Next, from equality (20), taking into account that the minimum value of the function on the positive semi-axis equals , we obtain that . Besides, is an increasing function on , and the justification of the inequality can be completed by induction:

Justification of this inequality completes the proof of the upper estimate of and in total the proof Theorem 3.

We note that the construction described in the proof of Theorem 3 can be straightforwardly adapted from the case of greedy expansions with prescribed coefficients to the generalized Approximate Weak Greedy Algorithm (see [10, heorem 3]).

5. The Case of Finite-Dimensional Spaces

In this section we prove the following theorem.

Theorem 5. Let the space be finite-dimensional, and let the coefficients converge to zero monotonically and satisfy the condition . Then for every symmetric unit-normed dictionary and any all realizations of greedy expansion of in with the prescribed coefficients and the weakness parameter converge to .

Thus in the finite-dimensional case at least for in the natural class of monotonic coefficient sequences conditions sufficient for the convergence of a greedy expansion to an expanded element include only the convergence of coefficients to zero and infinity of their sum. Clearly, convergence of coefficients to zero is also a necessary condition for the convergence of greedy expansion.

As the first step of the proof we note that due to Lemma 4 there exists a limit of norms of remainders of a greedy expansion with prescribed coefficients . Let us suppose that this limit is non-zero. Then starting from a certain index remainder norms are separated from zero. Due to the compactness of a sphere in a finite-dimensional space and the completeness and the symmetric property of the dictionary it implies that scalar products

are also separated from zero. In other words, there exist a positive number and an index such that for all the inequality holds. In addition there exists such an index that for all coefficient does not exceed . But consequently for it follows from equality (10) that

Hence

This contradiction completes the proof of Theorem 5.

6. Conclusion

The main results of the paper state that in Hilbert spaces a greedy expansion with prescribed coefficients converges to an expanded element if coefficients satisfy certain relatively weak conditions that do not include monotonicity, and these conditions can not be essentially relaxed. At the same time we showed that for the finite-dimensional case the relaxation is possible.

Data Availability

No data were used to support this study.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this article.

Acknowledgments

We thank Prof. T.P. Lukashenko and Dr. E.D. Livshits for valuable comments and discussions. The research was supported by the Russian Federation Government Grant No. 14.W03.31.0031.

References

B. S. Kashin and A. A. Saakyan, Orthogonal series, vol. 75 of Translations of Mathematical Monographs, American Mathematical Society, Providence, Rhode Island, RI, USA, 1989.

V. N. Temlyakov, “Weak greedy algorithms,” Advances in Computational Mathematics, vol. 12, no. 2-3, pp. 213–227, 2000.

V. N. Temlyakov, “Greedy expansions in Banach spaces,” Advances in Computational Mathematics, vol. 26, no. 4, pp. 431–449, 2007.

J. H. Friedman and W. Stuetzle, “Projection pursuit regression,” Journal of the American Statistical Association, vol. 76, no. 376, pp. 817–823, 1981.

L. K. Jones, “A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training,” The Annals of Statistics, vol. 20, no. 1, pp. 608–613, 1992.

S. G. Mallat and Z. Zhang, “Matching pursuits with time-frequency dictionaries,” IEEE Transactions on Signal Processing, vol. 41, no. 12, pp. 3397–3415, 1993.

V. Temlyakov, “Greedy algorithms with prescribed coefficients,” Journal of Fourier Analysis and Applications, vol. 13, no. 1, pp. 71–86, 2007.

V. V. Galatenko, “Expansion in a system of functions with fixed coefficients,” Moscow University Mathematics Bulletin, vol. 53, no. 1, pp. 12–15, 2003.

O. Rassudova, Zhadnye razlozhenija v gil’bertovyh prostranstvah [Greedy expansions in Hilbert spaces; in Russian], Materialy Voronezhskoj zimnej matematicheskoj shkoly “Sovremennye metody teorii funkcij i smezhnye problemy” [Proc. Voronezh Winter Mathematical School, Contemporary Methods in Theory of Functions and Adjacent Problems], Voronezh: VSU Publishing House, pp. 151–152, 2009.

V. V. Galatenko and E. D. Livshits, “Generalized approximate weak greedy algorithms,” Mathematical Notes, vol. 78, no. 1-2, pp. 170–184, 2005.