Well-Posedness and Primal-Dual Analysis of Some Convex Separable Optimization Problems

We focus on some convex separable optimization problems, considered by the author in previous papers, for which problems, necessary and sufficient conditions or sufficient conditions have been proved, and convergent algorithms of polynomial computational complexity have been proposed for solving these problems.The concepts ofwell-posedness of optimization problems in the sense of Tychonov, Hadamard, and in a generalized sense, as well as calmness in the sense of Clarke, are discussed. It is shown that the convex separable optimization problems under consideration are calm in the sense of Clarke. The concept of stability of the set of saddle points of the Lagrangian in the sense of Gol’shtein is also discussed, and it is shown that this set is not stable for the “classical” Lagrangian. However, it turns out that despite this instability, due to the specificity of the approach, suggested by the author for solving problems under consideration, it is not necessary to use modified Lagrangians but only the “classical” Lagrangians. Also, a primal-dual analysis for problems under consideration in view of methods for solving them is presented.


Statement of Problems under Consideration: Preliminary
Results.In this paper, we study well-posedness and present primal-dual analysis of some convex separable optimization problems, considered by the author in previous papers.For the sake of convenience, in this subsection we recall main results of earlier papers that are used in this study.
Assumptions for problem () are as follows.
Under these assumptions, the following characterization theorem (necessary and sufficient condition) for problem () was proved in [1].
Denote by ℎ ≤  , ℎ =  , ℎ ≥  the values of   , for which    (  ) = 0 for the three problems under consideration in this paper, respectively.
Theorem 1 (characterization of the optimal solution to problem ()).Under the above assumptions, a feasible solution x * = ( *  ) ∈ ∈  is an optimal solution to problem () if and only if there exists a  ∈ R The following polynomial algorithm for solving problem () with strictly convex differentiable functions   (  ),  ∈ , was suggested in [1].
It is proved in [1] that this algorithm is convergent.
Assumptions for problem ( = ) are as follows.
Under these assumptions, the following characterization theorem (necessary and sufficient condition) for problem ( = ) is proved in [2].
Theorem 4 (characterization of the optimal solution to problem ( = )).A feasible solution x * = ( *  ) ∈ ∈   is an optimal solution to problem ( = ) if and only if there exists a  ∈ R 1 such that Assumptions for problem ( ≥ ) are as follows.
Under these assumptions, the following theorem (sufficient condition) for problem ( ≥ ) is proved in [2].
Theorem 5 (sufficient condition for optimal solution to problem ( ≥ )).Let  *  ,  ∈  be components of the optimal solution to problem  = .Then: The following polynomial algorithm for solving problem ( = ) with strictly convex differentiable functions   (  ) was suggested in [2].Algorithm 6.
It is proved in [2] that this algorithm is convergent.

Advances in Operations Research
The following algorithm for solving problem ( ≥ ) with strictly convex differentiable functions   (  ) is suggested in [2].

Organization of the Paper.
The rest of the paper is organized as follows.In Section 2, the concepts of wellposedness of optimization problems in the sense of Tychonov, Hadamard, and in a generalized sense, as well as calmness in the sense of Clarke, are discussed.It is shown in Section 2.3 that the convex separable optimization problems under consideration are calm in the sense of Clarke.In Section 3, the concept of stability of the set of saddle points of the Lagrangian in the sense of Gol'shtein is also discussed and it is shown that this set is not stable for the "classical" Lagrangian.However, it is explained that despite this instability, due to the specificity of the approach, suggested by the author in previous papers for solving problems under consideration, it is not necessary to use modified Lagrangians but only the "classical" Lagrangians.In Section 4, primal-dual analysis of the problems under consideration in view of methods for solving them is presented.Main results of well-posedness and primal-dual analysis are included in Section 2.3 and in Sections 3 and 4.

Well-Posedness of Optimization Problems
Questions of existence of solutions and how they depend on problem's parameters are usually important for many problems of mathematics, not only in optimization.The term well-posedness refers to the existence and uniqueness of a solution and its continuous behavior with respect to data perturbations, which is referred to as stability.In general, a problem is said to be stable if where  is a given tolerance of the problem's data, () is the accuracy with which the solution can be determined, and () is a continuous function of .Besides these conditions, accompanying robustness properties in the convergence of sequence of approximate solutions are also required.Problems which are not well-posed are called ill-posed, or, sometimes, improperly posed.

Tychonov and Hadamard Well-Posedness
The problem ( 21) is Tychonov well-posed if and only if  has a unique global minimum point on  towards which every minimizing sequence converges.An equivalent definition is as follows: problem ( 21) is Tychonov well-posed if and only if there exists a unique x 0 ∈  such that (x 0 ) ≤ (x) for all x ∈  and There are two ways to cope with ill-posedness.The first one is to change the statement of the problem.The second one is the so-called Tychonov regularization.A parametric functional is constructed such that if it approaches 0, the solution of the "regularized" problem converges to the exact solution of the original problem.
Consider the problem Associate the following problem with (23): where   (x) is perturbation in the input data and x * (  (x)) is an optimal solution to the perturbed problem.Let when   → 0, then problem ( 23) is stable with respect to perturbation   (x).
A parametric function (x, Δ, (x)) with a parameter Δ is called a regularizing function for problem (23) with respect to perturbation   (x) if the following conditions are satisfied.
(2) If x * (  (x), Δ) is an optimal solution to problem then there exists a function when   → 0. Following Tychonov, an ill-posed problem is said to be regularizable if there exists at least one regularizing function for it.
The concept of Tychonov well-posedness can be extended to problems without the uniqueness of the optimal solution.Definition 10.Let  be a space with either a topology or a convergence structure associated, and  :  → R ≡ R ∪ {+∞} be a proper real-valued function.Problem ( 21) is said to be well-posed in the generalized sense if and only if arg min x∈ (x) ̸ = 0 and every sequence {u  } ⊂  such that (u  ) → inf{(x) : x ∈ } has some subsequence {k  } → u with u ∈ arg min x∈ (x).
Problem ( 21) is Tychonov well-posed if and only if it is well-posed in the generalized sense and arg min x∈ (x) is a singleton.
Hadamard well-posedness is primarily connected with problems of mathematical physics (boundary value problems for partial differential equations) and can be extended to mathematical programming problems.We do not discuss this topic here.
As recent studies in the calculus of variations, optimal control, and numerical methods of optimization show, uniqueness and continuity are often too restrictive to be adopted as the standards of well-posedness.It turns out that practical concepts concerning well-posedness are some forms of semicontinuity in the problem's data and solution mapping, along with potential multivaluedness in this mapping.

Calmness in the
for all points y, y  ∈ ; this is also referred to as a Lipschitz condition of rank .We say that  is Lipschitz (of rank ) near x if for some  > 0,  satisfies a Lipschitz condition (of rank ) on the set x +  (i.e., within an -neighborhood of x), where  is the open unit ball around 0.
A function , which satisfies a Lipschitz condition, sometimes is said to be Lipschitz continuous.
Let g and h be the functions g Let () be imbedded in a parametrized family (p, q) of mathematical programs, where p ∈ R  , q ∈ R  : (p, q) min  (x) Denote by  the feasible region of problem (p, q).Definition 12 (Clarke [3]).The value function  : R  × R  → R ∪ {±∞} is defined via (p, q) = inf{(p, q)} (i.e., the value of the problem (p, q)).If there are no feasible points for (p, q), then the infimum is over the empty set and (p, q) is assigned the value +∞.
Definition 13 (Clarke [3]).Let x solve ().The problem () is calm at x provided that there exist positive  and  such that for all (p, q) ∈ , for all x  ∈ x +  which are feasible for (p, q), one has where  is the open unit ball in  and ‖ (p, q) ‖ is the Euclidean norm of (p, q).
Let  be an open convex subset of .
Theorem 14 (Roberts and Varberg [4], Clarke [3]; Lipschitz condition from boundedness of a convex function).Let  be a convex function, bounded above on a neighborhood of some point of .Then, for any x in ,  is Lipschitz near x.
Recall that limit superior of a bounded sequence {  } in R, denoted lim sup{  } or lim{  }, equals the infimum of all numbers  ∈ R for which at most a finite number of elements of {  } (strictly) exceed .Similarly, limit inferior of {  } is given by lim inf{  } ≡ lim{  } ≡ sup{ : at most a finite number of elements of {  } are (strictly) less than }.
A bounded sequence always has a unique limit superior and limit inferior.
For p ∈ R  , let (p) be the infimum in the problem (p) in which the constraints   (x) ≤ 0 of problem () are replaced by   (x) +   ≤ 0.
Theorem 16 (Clarke [3]; Lipschitz property of the value function from Slater's condition).If  is bounded and  is Lipschitz on , then Slater's condition (i.e., the existence of a strictly feasible point) implies that  is Lipschitz near 0.
Theorems 15 and 16 mean that Slater's constraint qualification implies calmness of problem (p) in this case.
Theorem 17 (Clarke [3]; Calmness of a problem subject to inequality constraints).Let  incorporate only inequality constraints   (x) ≤ 0 and the abstract constraint x ∈  and suppose that the value function (p) is finite for p near 0.Then, for almost all p in a neighborhood of 0, the problem (p) is calm.
Remark 18.In the case of problem (), in which equality constraints exist, it is a consequence of Ekeland's theorem that (p, q) is calm for all (p, q) in a dense subset of any open set upon which  is bounded and lower semicontinuous.
Constraint qualifications (regularity conditions) can be classified into two categories: on the one hand, Mangasarian-Fromowitz and Slater-type conditions and their extensions, and, on the other hand, constraint qualifications called calmness.It turns out that calmness is the weakest of these conditions, since it is implied by all the others (see, e.g., Theorem 16).

Uniqueness of Solution.
The question of uniqueness of the optimal solution to problems under consideration is also important.
In the general case, if functions   (  ) are convex but not necessarily strictly convex, then, as it is known, a convex programming problem has more than one optimal solution and the set of optimal solutions to such a problem is convex.Further, the optimal value of the objective function is the same for all optimal solutions to problem () (problem ( = ) or problem ( ≥ ), resp.) if it has more than one optimal solution.If, for example, (6) (( 17), resp.) is a linear equation of  *  , then  *  ,  ∈   , are also uniquely determined from (6) (from (17), resp.).
Feasible regions of problems () and ( ≥ ) are nonempty by the assumption; this is satisfied when ∑ ∈   (  ) ≤  and ∑ ∈     ≥ , respectively.Without loss of generality, feasible regions Since the value function (p), associated with problems ((p)) and ( ≥ (p)), is finite near 0 (according to Definition 12 and the assumption that the corresponding feasible set is nonempty) then both problems are calm according to Theorem 17.
An alternative proof of calmness of problem () is the following.
The objective function (x) of problem () (1)-( 3) is convex (and, therefore, Lipschitz in accordance with Theorem 14), and Slater's constraint qualification is satisfied by the assumption.From Theorem 16, it follows that the value function (p) is Lipschitz, and problem () is calm at any solution x * of problem () according to Theorem 15.
Consider the parametrized family ( = (p, )) in which problem ( = ) is imbedded as follows: ( = (p, )) where   (p, ) is defined as follows: As it has been pointed out, whereas   (p, ) ̸ = 0 if Without loss of generality, assume that there exists a (p, ) such that   (p, ) ̸ = 0.This is satisfied, for example, when ∑ ∈     =  in addition to the requirement   ̸ = 0. Then the value function (p, ), associated with ( = (p, )), is finite by Definition 12.
Theorem 19 (Convexity of the infimum of a convex function subject to linear equality constraints).Let  be a convex function and  be a convex set in R  .Then, function We have used that  is a convex function, the property that and the fact that  ⊂  implies inf x∈ (x) ≤ inf x∈ (x).Therefore, ℎ(y) is a convex function by definition.
For problem ( = (p, )), matrix  of Theorem 19 consists of a single row, that is,  = 1, and convex set  is the dimensional parallelepiped The value function associated with problem ( = (p, )) is From Theorem 19 and the assumption that   (p, ) ̸ = 0, it follows that (p, ) is convex and finite, respectively, and from Theorem 14 it follows that it is Lipschitz.Then, problem ( = (p, )) is calm according to Theorem 15.
In the general case, if the mathematical program is not convex and equality constraints exist, we can use the approach of the Remark 18.Besides well-posedness of the optimization problems, stability of methods for solving these problems is also important.Let Φ(x, y) be a convex function of x ∈  and a concave function of y ∈ , where  and  are convex and closed sets.

On the
Recall the definition of a saddle point.
A point (x, ŷ) is said to be a saddle point of function Φ(x, y), x ∈ , y ∈ , if the following inequalities hold: for all x ∈ , y ∈ , that is, if This means that Φ(x, y) attains at the saddle point (x, ŷ) its maximum with respect to y for fixed x and Φ(x, y) attains at (x, ŷ) its minimum with respect to x for fixed ŷ. Set where (x, ) def = min z∈ ‖x − z‖ is the distance from x to the set .The implications written above mean that convergence of Φ(x, y * ) to Φ(x * , y * ) with respect to x ()  and convergence of Φ(x * , y) to Φ(x * , y * ) with respect to y () implies convergence of sequence ({x () }, {y () }) to the set  * ×  * of saddle points of Φ(x, y).
The concept of stability, introduced by Definition 20, is important for constructing iterative gradient algorithms for finding saddle points of the Lagrangian associated with an optimization problem.
The set of saddle points of the Lagrangian associated with the problem = {x ∈ R  :   (x) ≤ 0,  = 1, . . ., , x ∈ } (60) is not stable according to Definition 20.Concerning the dual variables this can be proved as follows.
is a nonempty set on which (x) > −∞, where dom  is the effective domain of .Otherwise,  is improper.Definition 9. Let  be a space with either a topology or a convergence structure associated and let  :  → R ≡ R ∪ {+∞} be a proper extended real-valued function.Consider the problem min  (x) subject to x ∈ .
Stability of the Set of Saddle Points of the Lagrangian 3.1.The Concept of Stability of Saddle Points of the Lagrangian.