Abstract and Applied Analysis

Volume 2014 (2014), Article ID 710158, 11 pages

http://dx.doi.org/10.1155/2014/710158

Research Article

Complexity Analysis of Primal-Dual Interior-Point Methods for Linear Optimization Based on a New Parametric Kernel Function with a Trigonometric Barrier Term

X. Z. Cai,¹ G. Q. Wang,² M. El Ghami,³ and Y. J. Yue¹

¹College of Advanced Vocational Technology, Shanghai University of Engineering Science, Shanghai 200437, China

²College of Fundamental Studies, Shanghai University of Engineering Science, Shanghai 201620, China

³Nesna University College, Mathematics Section, 8700 Nesna, Norway

Received 9 February 2014; Revised 15 May 2014; Accepted 18 May 2014; Published 15 June 2014

Academic Editor: Ngai-Ching Wong

Copyright © 2014 X. Z. Cai et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

We introduce a new parametric kernel function, which is a combination of the classic kernel function and a trigonometric barrier term, and present various properties of this new kernel function. A class of large- and small-update primal-dual interior-point methods for linear optimization based on this parametric kernel function is proposed. By utilizing the feature of the parametric kernel function, we derive the iteration bounds for large-update methods, , and small-update methods, . These results match the currently best known iteration bounds for large- and small-update methods based on the trigonometric kernel functions.

1. Introduction

In this paper, we consider the linear optimization (LO) problem in standard form

where with rank, , and . The dual problem of is given by

For years, LO has been one of the most active research areas in mathematical programming. There are many solution approaches for LO. Among them, the interior-point methods (IPMs) gain much more attention. Several efficient IPMs for LO and a large amount of results have been proposed. For an overview of the relevant results, see a recent book on this subject [1] and the references cited therein.

In the literature two types of primal-dual IPMs are distinguished: large-update methods and small-update methods, according to the value of the barrier-update parameter . However, there is still a gap between the practical behavior of these algorithms and these theoretical performance results. The so-called large-update IPMs have superior practical performance but with relatively weak theoretical results. While the so-called small-update IPMs enjoy the best known worst-case iteration bounds but their performance in computational practice is poor.

Recently, this gap was reduced by Peng et al. [2] who introduced the so-called self-regular kernel functions and designed primal-dual IPMs based on self-regular proximities for LO. They improved the iteration bound for large-update methods from to , which almost closes the gap between the iteration bounds for large- and small-update methods. Later, Bai, et al. [3] presented a large class of eligible kernel functions, which is fairly general and includes the classical logarithmic function and the self-regular functions, as well as many non-self-regular functions as special cases. The best known iteration bounds for LO obtained are as good as the ones in [2] for appropriate choices of the eligible kernel functions. Some well-known eligible kernel functions and the corresponding iteration bounds for large- and small-update methods are collected in Table 1. For some other related kernel-function based IPMs we refer to the recent books on this subject [4, 5].

Table 1: Complexity results for the eligible kernel functions.

The eligible kernel functions
Large-update methods
Small-update methods
Reference

1

For example [1]

2

[3]

3

[3]

4

[2]

5

[3]

6

[3]

7

[9]

8

[3]

9

[6]

10

[7]

11

[8]

12

[3]

13

[10]

Particularly, El Ghami et al. [6] first introduced a trigonometric kernel function for primal-dual IPMs in LO. They established the worst case iteration bounds for large- and small-update methods, namely, and , respectively. Peyghami et al. [7] considered a new kernel function with a trigonometric barrier term. Based on this kernel function, they proved that large-update method for solving LO has the worst case iteration bound, namely, , which improves the so far obtained iteration bound for large-update methods based on the trigonometric kernel function proposed in [6]. Recently, Peyghami and Hafshejani [8] established the better iteration bound for large-update methods based on a new kernel function consisting of a trigonometric function in its barrier term.

Motivated by their work, the purpose of this paper is to deal with the so-called primal-dual IPMs for LO based on a new kind of parametric kernel function as follows:

where (the bound of the is due to the proof of Lemma 3) and . We develop some new properties of the parametric kernel function, as well as the corresponding barrier function. Compared to the existing ones, the proposed function has a parameter . This implies that our kernel function includes a class of kernel functions. Furthermore, we present a class of primal-dual IPMs for LO based on this new parametric kernel function. The obtained iteration bound for large-update methods, namely, , which improves the classical iteration complexity with a factor , and for small-update methods, we derive the iteration bound, namely, , which matches the currently best known iteration bound for small-update methods.

The paper is organized as follows. In Section 2, we present the framework of kernel-based IPMs for LO. In Section 3, we introduce the new parametric kernel function with a trigonometric barrier term and develop some useful properties of the new kernel function, as well as the corresponding barrier function. The analysis and complexity of the algorithms for large- and small-update methods are presented in Section 4. Finally, Section 5 contains some conclusions and remarks.

Some notations used throughout the paper are as follows. , , and denote the set of vectors with components, the set of nonnegative vectors, and the set of positive vectors, respectively. denotes the 2-norm of the vector . denotes the identity vector. For any , (or ) denotes the smallest (or largest) value of the components of . Finally, if is a real valued function of a real nonnegative variable, the notation means that for some positive constant and means that for two positive constants and .

2. Framework of Kernel-Based IPMs for LO

In this section, we briefly recall the framework of kernel-based IPMs for LO, which includes the central path, the new search directions, and the generic primal-dual interior-point algorithm for LO.

2.1. Central Path for LO

Throughout the paper, we assume that both and satisfy the interior-point condition (IPC); that is, there exists such that

The Karush-Kuhn-Tucker (KKT) conditions for and are given by

where denotes the component-wise product of the vectors of and . The standard approach is to replace the third equation in (3), the so-called complementarity condition for and , by the parameterized equation , with . This yields the following system:

Since and the IPC holds, the parameterized system (4) has a unique solution for each . This solution is denoted as and we call the -center of and the -center of . The set of -centers (with running through all positive real numbers) gives a homotopy path, which is called the central path of and . If , then the limit of the central path exists and since the limit points satisfy the complementarity condition (3), the limit yields optimal solutions for and (see, e.g., [1]).

2.2. New Search Directions

IPMs follow the central path approximately and approach the optimal set of LO by letting go to zero. Applying Newton’s method to the system (4), we have

This system has a unique solution [2, 3]. Defining the vector

Note that the triple coincides with the -center if and only if . For further use we introduce the scaled search directions and according to

By using (6) and (7), after some elementary reductions, we have

where with .

It is obvious that the right-hand side in the third equation of the system (8) equals minus the derivative of the classic barrier function as follows:

where

is the kernel function of the classic barrier function. Thus, the system (8) can be rewritten as the following system:

Corresponding to the parametric kernel function (1), we define the barrier function as follows:

Due to the properties of the parametric kernel function , see, for example, Section 3, we can conclude that is a strictly convex function and attains minimal value at and ; that is,

Hence, the value of can be considered as a measure of the distance between the given iterate and the -center of the algorithms.

The approach in this paper differs only in one detail: we replace the right-hand side of the third equation in (8) by . This yields the following system:

The scaled search directions and are orthogonal vectors due to the fact that belongs to the null space and to the row space of the matrix . From (13), one may easily verify that the right-hand side in the system (14) vanishes if and only if . Thus we conclude that , , and all vanish if and only if , that is, if and only if , , and . Otherwise, we will use as the new search direction.

For the analysis of the interior-point algorithm, we define the norm-based proximity measure as follows:

One can easily verify that

2.3. Generic Primal-Dual Algorithm for LO

In general each kernel function gives rise to a primal-dual interior-point algorithm. Without loss of generality we assume that is known for some positive . For example, due to the above assumption we may assume this for , with . Then, we decrease to for some . We solve the scaled Newton system (14) and through (7) to get the new search direction . The new triple is given by

where denotes the default step size, , which has to be chosen appropriately. If necessary, we repeat the procedure until we find iterates that are in the neighborhood of . Then is again reduced by the factor and we apply Newton’s method targeting the new -centers, and so on. This process is repeated until is small enough, say until ; at this stage we have found an -solution of and . The generic form of this algorithm is shown in Algorithm 1.

Input:

    A threshold parameter ;

    an accuracy parameter ;

    a fixed barrier update parameter ;

begin

    ;

    while    do

    begin

        ;

        while    do

        begin

            calculate the search direction ;

            determine the default step size ;

            update .

        end

    end

end

Algorithm 1: Generic primal-dual interior-point algorithm for LO.

3. New Parametric Kernel Function and Its Properties

In this section, we introduce the new parametric kernel function with a trigonometric barrier term and develop some useful properties of the new kernel function as well as the corresponding barrier function that are needed in the analysis of the algorithms.

For ease of reference, we give the first three derivatives of given by (1) with respect to as follows:

where

One can easily verify that

This implies that the kernel function is completely defined by its second derivative as follows:

In what follows, we develop some technical lemmas on the parametric kernel function.

Lemma 1. Let . Then

Proof. Define . For , we have

It follows that

Thus is monotone decreasing in , and since , this implies the lemma.

Lemma 2. Let be a constant, and

Here are the functions of parameter for . If , and for , then we have for all .

Proof. It is obvious that , for all . This implies that is monotone increasing. Since , we have for all . And so on, we can conclude that for all . This completes the proof of the lemma.

Lemma 3. Let . Then

Proof. Firstly, we consider two cases to prove (28).

Case  1. Assume that . Then we have . Since , from (19) one can see that for all , when .

Case  2. Assume that . Define

We need to prove that when , holds. Using the fact that for all , we have

where

From Lemma 2, by solving for , we can conclude that

which implies that for all , when .

Secondly, we consider three cases to prove that when , (29) holds. We have

Case  1. Assume that . Since , we have and ; therefore holds for this case.

Case  2. Assume that . ;  therefore also holds for this case.

Case  3. Assume that . Using Lemma 1, we have

This implies that for all .

From three cases above we conclude that when , we have for all .

Thirdly, we consider two cases to prove that when , (30) holds.

Case  1. Assume that . We have and ; therefore holds for this case.

Case  2. Assume that . Then using the fact that and , we have

where

From Lemma 2, by solving for , we can conclude that

which implies that for all , when .

Finally, we consider three cases to prove that (31) holds while . We have

Define

Case  1. Assume that . In this situation, we have , , , and , from which we know that . Therefore holds for this case.

Case  2. Assume that . Due to the fact that , , and for all , we have

where

From Lemma 2, by solving for , we can conclude that

which implies that for all , when .

Case  3. Assume that . Due to the fact that , , and for all , we have

where

From Lemma 2, by solving for , we can conclude that

this implies that when , we have for all .

From the above discussions, the proof of the lemma is completed.

The property described below is exponential convexity, which has been proven to be very useful in the analysis of primal-dual interior-point algorithms based on the eligible kernel functions [2, 3].

Lemma 4. Let . Then

Proof. The result of the lemma follows immediately from Lemma  1 in [2], which states that the above inequality holds if and only if for all . Hence, from (29) in Lemma 3, the proof of the lemma is completed.

From (28) of Lemma 3 (i.e., ), we say that is strongly convex. The following lemma provides an important consequence of this property. These results can be directly obtained from the corresponding results in [3].

Lemma 5. Let . Then

As the consequences of Lemma 5, one has the following two corollaries.

Corollary 6. Let . Then

Corollary 7. Let . Then

Lemma 8. Let . Then

Proof. Let

Then

As , to prove the lemma, it is sufficient to show that the function is a decreasing function. For this purpose, we have

If , then , so .

If , then . Using , we have

This implies the result of the lemma.

Theorem 9. Let and . Then

Proof. It follows from Lemma 8 with that

Thus, we have, by Corollary 7,

The proof of the theorem is completed.

4. Analysis and Complexity of the Algorithms

In this section, we first choose a default step size. Then, we derive an upper bound for the decrease of the barrier function during an inner iteration. Finally, the iteration bounds for large- and small-update methods are established.

4.1. Computation of the Default Step Size

In each inner iteration, we first compute the search direction from the system (14). Then through (7), we obtain the search direction . Note that during an inner iteration the parameter is fixed. Hence, after the step the new -vector is given by

It follows from (17) that

Also using , we obtain

We consider the decrease in as a function of and define

However, working with may not be easy because in general is not convex. Thus, we are searching for the convex function that is an upper bound of and whose derivatives are easier to calculate than those of .

Lemma 4 implies that

Let

This makes clear that is an upper bound of . Furthermore, we can easily verify that .

Taking the derivative with respect to , we have

This gives, also using the third expression of system (14),

Differentiating once again, we get

Below we use the shorthand notation: . The following lemma provides an upper bound of , which can be found in Lemma  4.1 in [3].

Lemma 10. One has .

Following the strategy considered in [3], we briefly recall how to choose the default step size. Suppose that the step size satisfies

Then . The largest possible value of the step size of satisfying (71) is given by

where is the inverse function of for . Furthermore, we can conclude that

Since is the inverse function of , one has

This implies

for . Hence, putting , we get . Thus one has

From Lemma 1, it yields

which implies that

It also means that

Since and for all , one has

Therefore, using Corollary 6 (i.e., ), one has

Define

Then, we have

In the sequel, we use

as the default step size, which essentially depends only on the norm and on the constant .

4.2. Decrease of the Barrier Function during an Inner Iteration

In what follows, we will show that the barrier function in each inner iteration with the default step size , as defined by (84), is decreasing. For this, we need the following technical result.

Lemma 11 (Lemma 12 in [2]). Let be a twice differentiable convex function with , and let attain its (global) minimum at . If is increasing for , then

As a consequence of Lemma 11 and the fact that , which is a twice differentiable convex function with and , we can easily prove the following lemma.

Lemma 12. Let the step size be such that . Then .

The following theorem shows that the default step size (84) yields a sufficient decrease of the barrier function during each inner iteration.

Lemma 13. Let be the default step size as given by (84). Then

Proof. From Lemma 12, (84), and Corollary 6, we have

This completes the proof of the theorem.

4.3. Iteration Bounds for Large- and Small-Update Methods

From Theorem 9, after updating the parameter to with , we have

At the start of an outer iteration and just before updating the parameter , we have . Due to (88), the value of exceeds the threshold after updating . Therefore, we need to count how many inner iterations are required to return to the situation where . We denote the value of after the -update as and the subsequent values in the same outer iteration are denoted as , , where denotes the total number of inner iterations in the outer iteration. Hence, we have

According to a decrease of in Lemma 13, we have

where and .

Lemma 14 (Lemma 14 in [2]). Let be a sequence of positive numbers such that

where and . Then .

The following lemma provides an estimate for the number of inner iterations between two successive barrier parameter updates, in terms of and the constant .

Lemma 15. One has

Proof. Using (90) and also applying Lemma 14, the result of the lemma follows.

The number of outer iterations is bounded above by (cf., [1, .17, page 116]). By multiplying the number of outer iterations and the number of inner iterations, we get an upper bound for the total number of iterations, namely,

Then, the iteration bound for large-update methods is established in the following theorem.

Theorem 16. For large-update methods, one takes for a constant (independent on ), namely, and . The iteration bound then becomes

which improves the classical iteration bound with a factor . Similar to the analysis in [10], the iteration complexity for the small-update methods is straight and we leave it for the interested readers.

Theorem 17. For small-update methods, one takes and . The iteration bound then becomes

which matches the currently best known iteration bound for small-update methods.

5. Conclusions and Remarks

In this paper, we have proposed a class of primal-dual IPMs for LO based on a new parametric kernel function, which is a combination of the classic kernel function and a trigonometric barrier term. The worst case iteration bounds for large- and small-update methods are established, namely, and , respectively. For both versions of the kernel-based IPMs, the obtained iteration bounds match the currently best known iteration bound for such methods based on the trigonometric kernel functions.

The paper improved the complexity results for large-update methods obtained by El Ghami et al. [6] and generalized the results presented recently by Peyghami et al. [7]. Furthermore, the analysis deviates significantly from the analysis presented in previous papers [6, 7].

Some interesting topics for further research remain. The extension to second-order cone optimization (SOCO), semidefinite optimization (SDO), and symmetric cone optimization (SCO) deserves to be investigated. Furthermore, the numerical results may help us compare the behavior of the proposed algorithms with other existing IPMs.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

The authors would like to thank the anonymous referees for their useful comments and suggestions, which helped improve the presentation of this paper. This work was supported by Connotative Construction Project of Shanghai University of Engineering Science (no. NHKY-2013-08).

References

C. Roos, T. Terlaky, and J.-Ph. Vial, Theory and Algorithms for Linear Optimization. An Interior-Point Approach, Theory and Algorithms For Linear Optimization, Springer, John Wiley & Sons, Chichester, UK, 1st edition, 1997.

J. Peng, C. Roos, and T. Terlaky, “Self-regular functions and new search directions for linear and semidefinite optimization,” Mathematical Programming B, vol. 93, no. 1, pp. 129–171, 2002.

Y. Q. Bai, M. El Ghami, and C. Roos, “A comparative study of kernel functions for primal-dual interior-point algorithms in linear optimization,” SIAM Journal on Optimization, vol. 15, no. 1, pp. 101–128, 2004.

Y. Q. Bai, Kernel Function-Based Interior-Point Algorithms For Conic Optimization, Science Press, Beijing, China, 2010.

M. El Ghami, A Kernel Function Approach For Interior Point Methods: Analysis and Implementation, LAP Lambert Academic, Germany, 2011.

M. El Ghami, Z. A. Guennoun, S. Bouali, and T. Steihaug, “Interior-point methods for linear optimization based on a kernel function with a trigonometric barrier term,” Journal of Computational and Applied Mathematics, vol. 236, no. 15, pp. 3613–3623, 2012.

M. R. Peyghami, S. F. Hafshejani, and L. Shirvani, “Complexity of interior-point methods for linear optimization based on a new trigonometric kernel function,” Journal of Computational and Applied Mathematics, vol. 255, no. 1, pp. 74–85, 2014.

M. R. Peyghami and S. F. Hafshejani, “Complexity analysis of an interior-point algorithm for linear optimization based on a new proximity function,” Numerical Algorithms, 2014.

K. Amini and A. Haseli, “A new proximity function generating the best known iteration bounds for both large-update and small-update interior-point methods,” ANZIAM Journal, vol. 49, no. 2, pp. 259–270, 2007.

Y. Q. Bai, G. Lesaja, C. Roos, G. Q. Wang, and M. El Ghami, “A class of large-update and small-update primal-dual interior-point algorithms for linear optimization,” Journal of Optimization Theory and Applications, vol. 138, no. 3, pp. 341–359, 2008.