^{1}

^{2}

^{3}

^{1}

^{2}

^{3}

In this paper, we propose an accelerated proximal point algorithm for the difference of convex (DC) optimization problem by combining the extrapolation technique with the proximal difference of convex algorithm. By making full use of the special structure of DC decomposition and the information of stepsize, we prove that the proposed algorithm converges at rate of

Difference of convex problem (DCP) is an important kind of nonlinear programming problems in which the objective function is described as the difference of convex (DC) functions. It finds numerous applications in digital communication system [

It is well known that the method to solve the DCP is the so-called difference of the convex algorithm (DCA) [

In this paper, inspired by the work in [

The remainder of the paper is organized as follows. In Section

To end this section, we recall some definitions used in the subsequent analysis [

For an extended real valued function

Furthermore, if

Consider the following difference of convex programming:

For the DCP, the following is a classical DCA which takes the following iterative scheme [

By replacing the concave part in the objective function by a linear majorant and replacing the smooth convex part by a quadratic majorant, Gotoh et al. [

where

Despite a simple subproblem is involved in the algorithm, the PDCA is potentially slow [

In this section, we establish the global convergence of the algorithm and its convergence rate. To continue, we first recall the following conclusions.

(see [

Let

Since

Connecting the fact that

It follows from

Connecting (

On the other hand, since

Connecting the fact that

Adding

By taking

By the optimality conditions of (

Then, for

Before proceeding further, we need the following conclusions.

(see [

Let

From (

Hence, to show the assertion, we only need to show that

In fact, by taking

Hence,

Using Lemma

Multiplying (

Now, we are ready to show the convergence rate of the APDCA.

For the sequence

Using the notations used in Lemma

Hence,

Then, from Lemma

Then, it follows from Lemma

The desired result follows.

In this section, we evaluate the performance of the APDCA by applying it to the DC regularized least squares problem. We will compare the performance of the APDCA with the algorithm in [

On APDCA and PDCA, we set

Furthermore, we terminate PDCA when the number of iteration is more than 5000 (denoted by “max” on the report).

Least squares problems with

This problem takes the form of (

To compare the performance of the three algorithms, we report the number of iterations (denoted by Iter), CPU times in seconds (denoted by CPU time), the sparsity of the solution (denoted by sparsity), and the function values at termination (denoted by fval), averaged over the 30 random instances. The numerical results are reported in Tables

Solving (

Iter | CPU time | ||||||
---|---|---|---|---|---|---|---|

GIST | APDCA | PDCA | GIST | APDCA | PDCA | ||

720 | 2560 | 1750 | 909 | Max | 3.57 | 1.38 | 7.37 |

1440 | 5120 | 1629 | 802 | Max | 13.7 | 5.0 | 31.8 |

2160 | 7680 | 1724 | 802 | Max | 28.5 | 10.0 | 62.2 |

2880 | 10240 | 1742 | 1002 | Max | 52.8 | 22.3 | 112.2 |

3600 | 12800 | 1799 | 1002 | Max | 83.8 | 34.3 | 174.7 |

4320 | 15360 | 1739 | 1002 | Max | 113.7 | 48.9 | 246.5 |

5040 | 17920 | 1778 | 1002 | Max | 160.7 | 66.9 | 334.5 |

5760 | 20480 | 1826 | 1002 | Max | 178.3 | 71.5 | 366.1 |

6480 | 23040 | 1778 | 975 | Max | 244.3 | 100.5 | 524.1 |

7200 | 25600 | 1752 | 975 | Max | 317.4 | 130.9 | 692.6 |

Sparsity | Fval | ||||||

GIST | APDCA | PDCA | GIST | APDCA | PDCA | ||

720 | 2560 | 783 | 761 | 1132 | 2.9755 | 2.9743 | 4.5442 |

1440 | 5120 | 1575 | 1614 | 2240 | 6.1144 | 6.1122 | 9.4466 |

2160 | 7680 | 2367 | 2424 | 3425 | 9.4648 | 9.4612 | 1.4594 |

2880 | 10240 | 3117 | 2910 | 4496 | 1.2312 | 1.2308 | 1.8319 |

3600 | 12800 | 3889 | 3644 | 5707 | 1.5896 | 1.5890 | 2.4309 |

4320 | 15360 | 4766 | 4376 | 6720 | 1.8879 | 1.8869 | 2.8401 |

5040 | 17920 | 5497 | 5141 | 7911 | 2.2523 | 2.2512 | 3.4175 |

5760 | 20480 | 6327 | 5931 | 9181 | 2.6870 | 2.6859 | 4.1224 |

6480 | 23040 | 7065 | 6716 | 10184 | 2.9070 | 2.9098 | 4.3889 |

7200 | 25600 | 7865 | 7449 | 11286 | 3.2206 | 3.2191 | 4.8588 |

Solving (

Iter | CPU time | ||||||
---|---|---|---|---|---|---|---|

GIST | APDCA | PDCA | GIST | APDCA | PDCA | ||

720 | 2560 | 972 | 591 | Max | 1.5 | 0.7 | 5.4 |

1440 | 5120 | 968 | 602 | Max | 6.1 | 2.8 | 23.2 |

2160 | 7680 | 993 | 602 | Max | 13.6 | 6.1 | 50.2 |

2880 | 10240 | 835 | 602 | Max | 19.8 | 10.6 | 88.6 |

3600 | 12800 | 973 | 602 | Max | 36.1 | 16.7 | 139.8 |

4320 | 15360 | 931 | 602 | Max | 49.2 | 23.5 | 202.5 |

5040 | 17920 | 941 | 602 | Max | 67.5 | 32.6 | 296.4 |

5760 | 20480 | 979 | 602 | Max | 100.7 | 43.5 | 354.9 |

6480 | 23040 | 992 | 602 | Max | 116.3 | 54.9 | 449.8 |

7200 | 25600 | 939 | 602 | Max | 138.0 | 67.4 | 558.5 |

Sparsity | Fval | ||||||

GIST | APDCA | PDCA | GIST | APDCA | PDCA | ||

720 | 2560 | 728 | 703 | 927 | 6.2438 | 6.2430 | 7.6433 |

1440 | 5120 | 1449 | 1381 | 1838 | 1.3160 | 1.3159 | 1.6346 |

2160 | 7680 | 2168 | 2086 | 2810 | 2.0060 | 2.0058e | 2.5146 |

2880 | 10240 | 2853 | 2745 | 3618 | 2.3976 | 2.3973 | 2.7654 |

3600 | 12800 | 3675 | 3557 | 4607 | 3.0264 | 3.0260 | 3.5620 |

4320 | 15360 | 4368 | 4195 | 5523 | 3.9802 | 3.9798 | 4.7740 |

5040 | 17920 | 5132 | 4925 | 6501 | 4.7413 | 4.7407 | 5.7676 |

5760 | 20480 | 5825 | 5656 | 7358 | 5.3208 | 5.3202 | 6.3891 |

6480 | 23040 | 6597 | 6311 | 8361 | 5.7707 | 5.7699 | 6.9385 |

7200 | 25600 | 7270 | 7052 | 9269 | 6.4648 | 6.4640 | 7.7325 |

Least squares problems with logarithmic regularizer are as follows:

This problem takes the form of (

To compare the performance of the three algorithms, we report the number of iterations (denoted by Iter), CPU times in seconds (denoted by CPU time), the sparsity of the solution (denoted by sparsity), and the function values at termination (denoted by fval), averaged over the 30 random instances. The numerical results are reported in Tables

Solving (

Iter | CPU time | ||||||
---|---|---|---|---|---|---|---|

GIST | APDCA | PDCA | GIST | APDCA | PDCA | ||

720 | 2560 | 843 | 596 | Max | 1.6 | 0.7 | 5.5 |

1440 | 5120 | 672 | 602 | Max | 5.6 | 3.0 | 22.3 |

2160 | 7680 | 873 | 602 | Max | 12.4 | 6.1 | 49.5 |

2880 | 10240 | 876 | 602 | Max | 21.2 | 10.6 | 87.4 |

3600 | 12800 | 871 | 602 | Max | 32.7 | 16.6 | 138.0 |

4320 | 15360 | 845 | 602 | Max | 45.1 | 23.4 | 194.8 |

5040 | 17920 | 872 | 602 | Max | 62.5 | 32.0 | 265.6 |

5760 | 20480 | 846 | 602 | Max | 79.4 | 41.4 | 345.3 |

6480 | 23040 | 877 | 602 | Max | 104.1 | 52.8 | 441.0 |

7200 | 25600 | 816 | 602 | Max | 120.4 | 66.0 | 547.5 |

Sparsity | Fval | ||||||

GIST | APDCA | PDCA | GIST | APDCA | PDCA | ||

720 | 2560 | 705 | 661 | 931 | 3.8979 | 3.8973 | 5.6815 |

1440 | 5120 | 1395 | 1345 | 1794 | 7.1306 | 7.1293 | 9.3006 |

2160 | 7680 | 2123 | 2011 | 2710 | 1.1455 | 1.1453 | 1.5861 |

2880 | 10240 | 2809 | 2705 | 3597 | 1.4878 | 1.4876 | 2.0601 |

3600 | 12800 | 3570 | 3418 | 4503 | 1.9187 | 1.9182 | 2.7236 |

4320 | 15360 | 4277 | 4103 | 5370 | 2.3163 | 2.3159 | 3.1699 |

5040 | 17920 | 5042 | 4729 | 6287 | 2.6491 | 2.6486 | 3.6295 |

5760 | 20480 | 5689 | 5501 | 7199 | 3.0649 | 3.0643 | 4.3049 |

6480 | 23040 | 6353 | 6093 | 8057 | 3.4115 | 3.4110 | 4.7749 |

7200 | 25600 | 7139 | 6089 | 8924 | 3.7435 | 3.7427 | 5.1004 |

Solving (

Iter | CPU time | ||||||
---|---|---|---|---|---|---|---|

GIST | APDCA | PDCA | GIST | APDCA | PDCA | ||

720 | 2560 | 497 | 329 | 4658 | 0.9 | 0.4 | 5.2 |

1440 | 5120 | 468 | 402 | 4582 | 3.1 | 1.9 | 20.5 |

2160 | 7680 | 496 | 402 | 4739 | 6.8 | 4.0 | 46.8 |

2880 | 10240 | 472 | 402 | 4527 | 11.1 | 7.0 | 79.5 |

3600 | 12800 | 494 | 402 | 4601 | 18.3 | 11.1 | 126.5 |

4320 | 15360 | 505 | 402 | 4602 | 26.6 | 16.5 | 179.0 |

5040 | 17920 | 451 | 402 | 4428 | 31.8 | 21.3 | 234.7 |

5760 | 20480 | 448 | 402 | 4446 | 41.2 | 27.7 | 304.2 |

6480 | 23040 | 459 | 402 | 4602 | 52.8 | 35.0 | 403.6 |

7200 | 25600 | 487 | 402 | 4668 | 70.7 | 44.0 | 510.4 |

Sparsity | Fval | ||||||

GIST | APDCA | PDCA | GIST | APDCA | PDCA | ||

720 | 2560 | 628 | 635 | 658 | 7.5032 | 7.5032 | 7.5053 |

1440 | 5120 | 1300 | 1248 | 1337 | 1.4892 | 1.4891 | 1.4896 |

2160 | 7680 | 1987 | 1865 | 1965 | 2.3348 | 2.3347 | 2.3354 |

2880 | 10240 | 2543 | 2462 | 2627 | 3.0410 | 3.0410 | 3.0416 |

3600 | 12800 | 3156 | 3072 | 3252 | 3.8829 | 3.8828 | 3.8837 |

4320 | 15360 | 3831 | 3703 | 3973 | 4.5346 | 4.5344 | 4.5348 |

5040 | 17920 | 4460 | 4300 | 4605 | 5.2664 | 5.2662 | 5.2676 |

5760 | 20480 | 5124 | 4991 | 5268 | 5.9404 | 5.9402 | 5.9417 |

6480 | 23040 | 5761 | 5540 | 5919 | 6.8740 | 6.8737 | 6.8756 |

7200 | 25600 | 6365 | 6231 | 6632 | 7.6681 | 7.6678 | 7.6700 |

In this paper, we propose an accelerated proximal point algorithm for the difference of convex optimization problem by combining the extrapolation technique with the proximal difference of the convex algorithm. By making full use of the special structure of difference of convex decomposition and the information of stepsize, we prove that the proposed algorithm converges at rate of

The data used to support the findings of this study are available from the corresponding author upon request.

The authors declare that they have no conflicts of interest.

The authors equally contributed to this paper and read and approved the final manuscript.

This project was supported by the Natural Science Foundation of China (grants nos. 11801309, 11901343, and 12071249).