Linear Programming with Inequality Constraints via Entropic Perturbation

A dual convex programming approach to solving linear programs with inequality constraints through entropic perturbation is derived. The amount of perturbation required depends on the desired accuracy of the optimum. The dual program contains only non-positivity constraints. An e-optimal solution to the linear program can be obtained effortlessly from the optimal solution of the dual program. Since cross-entropy minimization subject to linear inequality constraints is a special case of the perturbed linear program, the duality result becomes readily applicable. Many standard constrained optimization techniques can be specialized to solve the dual program. Such specializa-tions, made possible by the simplicity of the constraints, significantly reduce the computational effort usually incurred by these methods. Immediate applications of the theory developed include an entro-pic path-following approach to solving linear semi-infinite programs with an infinite number of inequality constraints and the widely used entropy optimization models with linear inequality and/or equality constraints.

The motivation of this study is twofold. First, Fang and Wu [9] recently proposed an entropic path-following approach to solving linear semi-infinite programs with finitely many variables and infinitely many inequality constraints. Their algorithms require solving an entropically perturbed linear program with finitely many inequality constraints. After introducmg artificial variables, the re~ulting equality-constrained convex program is no longer an entropically perturbed linear program due to the absence of the entropic terms for the artificial variables. Therefore, the algorithms pro posed in [7] is no longer applicable and an algorithm for solving directly the entropically perturbed linear programs with inequality constraints is needed. Second, the widely applicable entropy optimi zation problem with linear inequality constraints turns out to be a special case of the perturbed linear program being treated. Although such minimization problems subject to equality constramts have been used widely and treated extensively in recent literature [e.g. [10][11][12][13][14][15][16], the inequality case has received little attention. Nevertheless, the inequality formulation is particularly appealing when point estimates for the linear moments of the underlying distribution, i.e. the right-hand sides of the equal Ity formulation, cannot be accurately obtained but the interval (range) estimates for the moments are available.
In this paper, we extend the geometric programming approach to derive the dual program in Section 2, discuss other applications of the duality results in Section 3, and conclude the paper in S,ection 4.

A DUAL APPROACH WITH ENTROPIC PERTURBATION.
Consider the following (primal) linear program: where c and x are n-dimensional column vectors, A is an m x n (m s; n) matrix, b is an m dimensional column vector, and 0 is the n-dimensional zero column vector.
The linear dual of Program P is given as follows: Program D: Maximize bTw subject to ATw s; c where w is an m-dimensional column vector.
Following the approach developed in [5], for any given scalar 1.1 > 0, instead of solving Program P directly, we tackle the following nonlinear program with an entropic perturbation: Note that this inequality become~ an equahty if and only tf z= I. Recall that the right-hand side of (2.8) is exactly the objective function of Program Pw We now define the following geometric dual program D 11 of P 11 : Program D 11 can also be derived via the Lagrangian approach. Note that this dual program differs from the one obtained for standard-form linear programs in [7] only in the extra non-positivity requirements. While it is usually the case and easy to see that, in the Lagrangian max-min denva twn, a change of stgn m a primal constraint re,ults m a change of range of the corresponding dual variable, thts causal relationship i' not apparent in the geometric programming denvation. Our derivation, in contrast with its counterpart for the equality-constrained program, illustrates the difference in deriving the geometric dual program between the equality-constrained and the mcquality-constrained ca,es.
We now turn to establishing the duality theory.
With m=m 1 +m 2 and the notation wT=(w!,wJ), where w 1 is an m 1 -dimenstonal column vector and w2 is an m 2 -dimensional column vector, the geometric dual is defined as •=I J=l With the notation AT=(A!.AJ), we state the following theorem, whose proof is straightforward in light of the derivation provided above and treatment of the standard-form linear programs in [7].
THEOREM 5. If Program p' has an interior feasible solution, then Program D~, for every JDO, attains a finite maximum and Min(P~) = Max(D~). If, in addition, the constraint matrix A has full row-rank, then Program D~, for every JDO, has a umque optimal solution w*{J.L). In either case, equation (2.9) provides a dual-to-primal conversion which defines the optimal solution x*(J.L) of Pro gram P~.
As we stated before, if the feasible domain of Program P is bounded, then the optimal solution of Program P 11 converges to an optimal solution of Program P, as J.L reduces to zero. Actually, by simply modifying a parallel result in [7], we can easily construct an £-optimal solution according to the following theorem without any difficulty: THEOREM 6. If Program p' has an interior feasible solution x > 0 and its feasible domain is contained in a spheroid centered at the origin with a radius of M > 0, then, for any J.l > 0 such that J.l :5 E I 2n't , (2.15) where 't =max{ 1/e, IMlnMI }, (2.16) the optimal solution of Program P~ is an £-optimal solution of Program P'.

CROSS-ENTROPY MINIMIZATION SUBJECT TO INEQUALITY CONSTRAINTS
The cross-entropy minimization problem has received much attention in the recent literature [10][11][12][13][14][15][16]. However, most of the attention has focused on the case with equality constraints (in addi tion to the non-negativity constraints). In fact, a more general setting of linearly-constrained minimum cross-entropy problem can be described in the following form (assuming Pi> 0, j=1 ,2, ... ,n): Although the inequality constraint~ can be converted mto equality ones by adding ~lack vanable~. the rc~ulting program is no longer a regular entropy optimization problem due to the absence of the entroptc terms x}nxJ for the slack variables in the objective function. Therefore, the duality theory developed in [16] and the algorithms developed in [ 10] are not applicable. Also note that Program Q 1~ a ~pecial case of Program P~ with 1.1 = I and cJ =-In Pr Therefore, the theory developed m the prevtou~ ~ection applies readily to Program Q. In particular, the geomctnc dual program of Program Q can be derived as follows:

CONCLUSION
We have extended the unconstrained convex programming approach to solving linear programs with mequality constraints without adding artificial variables. By the duality theory, one can solve a given linear program by solving the geometric dual of a perturbed linear program. Many standard constrained optimization techniques [e.g. 18] can be specialized to solve the dual program D~. Such specializations, made possible by the simplicity of the constraints, significantly reduce the computa tional effort usually incurred by these methods. For example, the projection operation required by the projective gradient method is trivial, which makes the method a good candidate solution algo rithm.
Immediate applications of the theory developed include an entropic path-following approach to solving linear semi-infinite programs with an infinite number of inequality constraints and the widely used entropy optimization models with linear equality and/or inequality constraints.