Certificates of Optimality for Mixed Integer Linear Programming Using Generalized Subadditive Generator Functions

We introduce generalized subadditive generator functions for mixed integer linear programs. Our results extend Klabjan’s work frompure integer programswith nonnegative entries to generalMILPs.These functions suffice to achieve strong subadditive duality. Several properties of the functions are shown. We then use this class of functions to generate certificates of optimality for MILPs. We have performed a computational test study on knapsack problems to investigate the efficiency of the certificates.


Introduction
Most optimization problems in industry and science today are solved using commercial software packages.Such software packages are to some extent reliable and fast but they generally lack one important feature: verifiability of computations and/or optimality.Verification of reported benchmark results is very crucial in computational research [1].Commercial solvers rarely, if at all, release to the user the exact procedure that has been followed to solve a specific problem.Without this, on what basis does one trust a solution generated using these solvers?Even if one uses his/her own code for optimization, there are possibilities of bugs, mistakes, and so forth.
This issue becomes more apparent when we encounter MILP instances for which different solvers return different solutions as optimal.See [2] for such examples and related discussions.
The answer to this problem is to obtain a certificate of optimality: some information that proves the optimality of the solution at hand but is easier to check compared to the original optimization problem.
Suppose that a MILP has been solved using a solver.Verifying the optimality of the solution could be an arduous task.
Let us assume that we have used cutting planes to solve a MILP instance.In order to verify the optimality, we need to verify that each cutting plane that has been generated is indeed valid for MILP.Both numerical round-off errors and/or coding errors can cause generation of invalid inequalities.Verifying validity of a cut is equivalent to solving another MILP as hard as the original problem.If we use a branch and bound method, then verification of optimality is equivalent to a tree traversal and, in each node of the tree, the correctness of LP relaxation bound must be verified.This can get very complicated if a branch and cut method is used.
For example, see [3] where the authors provide a certificate of optimality for an optimal TSP tour through 85,900 cities.This certificate consists of traversing the branch and bound tree and checking optimality at each node and also checking the validity of each cutting plane that has been added.The time needed to check the optimality using this certificate might reach up to 568.9 hours (24 days).The actual time for solving the problem is 286.2 days.
On the other hand, a dual function with the same value as the objective value of the (doubtfully) optimal solution verifies the optimality immediately.In the case of linear programming, this is trivial since the dual vector provides such a tool for certification of optimality.However, in the case
The inequalities follow from subadditivity of  and Lemma 1.
Theorem 3 (see [8]).If the primal problem (MILP) has a finite optimum, then so does the dual problem (SD) and in this case they are equal.
Unlike linear programming, finding a dual optimal subadditive function does not seem to be straightforward.Two well-known families are known [9], but they appear to be difficult to work with.However, in the case when all the entries of  and  are nonnegative and all variables are required to be integers, Klabjan [4] defined a family of subadditive functions sufficient for strong duality that are computationally attractive.
In the case of linear programming, it is easy to see that the function  LP () = max V∈R  {V : V ≤ } is a feasible subadditive function and, using this function, the subadditive dual will reduce to the LP dual.
The subadditive dual plays an important role in the study of mixed integer programming.Any feasible solution to the dual gives a lower bound to the (MILP).A dual feasible function  with () =  IP is a certificate of optimality for the (MILP).It will be equivalent to the dual vector in LP and the reduced cost of column  can be defined as   −(  ).Many other properties of LP, such as complementary slackness, for example, can also be extended to MILP.Similar to LP, all optimal solutions can only have nonzero values for variables indexed by  with   = (  ) for an optimal  (see [4] for more details).

Generalized Subadditive Generator Functions
Klabjan [4] defined subadditive generator functions for pure integer programming problems with nonnegative entries as follows.
For a pure integer program min with nonnegative  and , a subadditive generator function is defined for a given  ∈ R  as where () = { ∈  :   >   }.Also, a subadditive ray generator function is defined for a given  ∈ R  as where () = { ∈  :   > 0}.
In [4], it is shown that these functions are feasible to the subadditive dual, and they are sufficient to achieve strong duality for an IP with nonnegative rational data.In this paper, we extend Klabjan's work to (MILP) with no restriction on input data.Let  ∈ R  and define () = { ∈  :   >   }.Let  ⊆  be such that () ⊆ .Also, let  ⊆ R  be such that Note that if  = R  and  = 0, then  ,, (ℓ) = ℓ.We will show in Section 5 that empty  means that the LP relaxation of (MILP) solves the (MILP) as well, so ℓ is an optimal LP dual vector.Also, if  = R  and  = () ̸ = 0, then  ,, is identically −∞.In general, one can choose  and  in many different ways.However, later in Section 5, we will show that, with specific choices of  and  ( = () ∪ { ∈  :   ̸ ≥ 0} and  = R  + ), we can generate good certificates for (MILP).Also, note that 0 ∈ .
For a polyhedron  ⊆ R  , the recession cone of  is defined as rec() = { ∈ R  : + ∈  for all  ∈ ,  ≥ 0}.We will need the following result of Meyer [10].
(2) If  is nonempty, the recession cones of conv() and  coincide.
Proof.First, we will show that  ,, (  ) ≤   for all  ∈ .We have There are two cases: for  ∈ ,  is either in  or not.If  ∈ , then  =   is a feasible solution to the maximization problem where   is the unit vector with 1 as the th component and zero elsewhere.This gives us  ,, (  ) ≤   .If  ∉ , then  ∉ (), so   ≤   which implies that  = 0 is a feasible solution to the maximization problem and gives us  ,, (  ) =   ≤   .
To show that  ,, (  ) ≤   for all  ∈ , first note that Gomory and Johnson [11] shows that if  ,, (ℓ) is finite, then the limsup and the ordinary limit coincide.Now, we have If  is in , then  = ℎ  is a feasible solution to the maximization problem where   is the unit vector.Then, the maximum will be greater than or equal to ℎ(  −   ), so the limit will be greater than or equal to   −   since ℎ > 0. This gives us  ,, (  ) ≤   .If  ∉ , then  = 0 is a feasible solution to the maximization problem and a similar argument gives us  ,, (  ) ≤   ≤   .
Proof.Let ℓ 1 , ℓ 2 ∈ ().To prove subadditivity, that is, If  * 1 and  * 2 are optimal solutions for two maximization problems on the left, then  * 1 +  * 2 is a feasible solution to the problem on the right and the result follows.
Proof.Without loss of generality, we can assume that  ≥ 0, since otherwise one can multiply rows of  =  with negative right hand side by −1 to make  nonnegative and then consider the new problem.
Let  = { :  ≥ 0}.Let (MILP) be feasible, and let    ≤   0 for  ∈  be valid inequalities for the set such that Finding such valid inequalities is possible by Theorem 5 where it is shown that if  and  in (MILP) are rational, then the convex hull of the feasible points is a polyhedron (finitely generated).Let (, ) be an optimal dual vector where  corresponds to constraints . Then, we have We show that, for these  and ,  ,, () ≥  * MILP .The dual program of the above LP is The optimal value of this problem is  * MILP .Let  be a nonnegative vector with   integer for  ∈  and ∑ ∈     ≤ .We have The has optimal value  * > 0. So there exists some  such that where Ẽ = { ∈ {1, . . ., } :   > 1} and  = { ∈  :   > 0} ∪ { ∈  :   ̸ ≥ 0}.Since  = 0 is feasible to the maximization problem in (22), we have and the proof is complete.

Properties of Generalized Subadditive Generator Functions
If  is any subadditive function with (0) = 0 and is dual feasible, then is a valid inequality for (MILP) [8].
In this section, we will show that, for any MILP, there is a finite set of generalized subadditive generator functions such that their corresponding valid inequalities give a finite description of the convex hull of the MILP.For this reason, it is enough to restrict our attention to a subset of subadditive generator functions called basic subadditive generator functions.
In this section,   denotes a submatrix of  with column indices in , and   is a subvector of  with indices in .
Theorem 10.The optimum value of (MILP) is equal to max{ : (, ) ∈   ()}, where for some  and .Proof. has a finite number of extreme rays.Let  denote the set of extreme rays of .Since  is finite, we have which is obviously a polyhedron (finitely generated).
It is obvious that one only needs basic subadditive generator functions.But since there are finite choices for  and   () has a finite number of extreme points, there are only a finite number of basic subadditive generator functions.So the following theorem holds.(1) LP ( 28) is infeasible if and only if (MILP) is infeasible. (

2) LP (28) has unbounded optimum value if and only if
(MILP) has unbounded optimum value.
(3) If neither of the cases above holds, then LP (28) has an optimal extreme point solution which is also optimal for (MILP).
If  ⊆ R  is a convex polyhedron, then a face of  is defined as  =  ∩ { ∈ R  :  = }, where  ≤  is a valid inequality for .A facet is a face with dimension dim() − 1.Since we know that the convex hull of feasible solutions to (MILP) is polyhedral, that is, it can be described by a finite set of facet defining valid inequalities, we have the following corollary.
Corollary 14.For  and  rational, there exists a finite set of subadditive generator functions {   ,, }  =1 such that is the convex hull of solutions to (MILP) with right hand side .

Certificates of Optimality and Sensitivity Analysis
In this section, we show that subadditive generator functions can be used as certificates of optimality for MILPs.Proof.Without loss of generality, assume that  ≥ 0. If  is such that  = 0, then  ,, () =  with  ≤ .This means that  is the optimal solution to the linear programming dual of the LP relaxation of (MILP).Conversely, suppose that  is the optimal solution to the linear programming dual of the LP relaxation.Then, we have  ,, () =  with  ≤  and obviously  = 0.
The following example shows that there might not be a unique optimal .Example 18.Consider the following IP: Note that if the size of  is much smaller than , then the certificate that we have is much easier to check since the number of variables is remarkably reduced.However, there still might be instances where  =  or size of  is comparable to .In this case, obviously, we do not have a good certificate.Fortunately, in our computational experiments, such cases are not observed for the families of problems considered.For example, consider the following pure ILP: Advances in Operations Research 7 Clearly, there is a unique optimal solution ( 1 =   = 1 and  2 = ⋅ ⋅ ⋅ =  −1 = 0).In this example,  * =  with  = {1, . . .,  − 1}.
Note that, in such cases where  has large cardinality, we still have a certificate; that is, we have a (subadditive) dual feasible function  with () =  * MILP .Verifying the optimality using this certificate is still less expensive than using the branch and bound tree as a certificate and verifying the validity of all cutting planes (as mentioned before, verification of validity of a cutting plane is equivalent to solving a mixed integer of the same size as the original MILP).
We refer the reader to [5,13] where the authors state the conditions under which primal feasibility, dual feasibility, and optimality still hold for  * and  * (optimal primal vector and subadditive dual function) after changes made to the input of (MILP).
In Definition 4,  can be any convex cone satisfying (5).However, choosing  to be the nonnegative orthant makes us able to generate better certificates.For the rest of the paper, we focus on subadditive generator functions with  = ()∪{ ∈  :   ̸ ≥ 0} and  = { :  ≥ 0} as stated previously in the proof of Theorem 9. Note that if we add the cut  ≥  * MILP to (MILP), then obviously we can use  calculated using Theorem 9 since for that  we have   () =  * =  * MILP .However, based on our empirical observations through computational experiments, the size of () in this case is usually equal to or comparable to .We have observed that the more cuts we add to the LP relaxation of (MILP), the better  we can get using Theorem 9. We have also observed that if we add enough cuts so that the optimal solution of the LP relaxation is  * , then the size of  is much less than , so we can consider it as a good certificate.
We should note that the cutting planes    ≤   0 ,  ∈ , must be valid inequalities for the set  = { ∈ Z  + × R  + :  ≤ } by Theorem 9.This leads us to the lifting problem: each cutting plane that we add to the LP relaxation of (MILP) must be lifted first to become valid for the set .
Next, we show that the set {(, ) ∈ Z  + × R  + :  +  = } ̸ = 0 is a face of {(, ) ∈ Z  + × R  + :  +  ≤ }.So the problem of lifting is well defined.In other words, the following theorem shows that the feasible region of (MILP) is a face of the polyhedral set .
Theorem 20.The polyhedral set is a face of the polyhedral set where  and  have rational entries.
In order to prove this theorem, we first need few lemmas.
Assume that (MILP) has been solved to optimality using a cutting plane method and that we have access to all the cutting planes that have been added to the LP relaxation of (MILP).Now, we can use a standard method mentioned in [15] by Espinoza et al. to lift all the cuts that we need and then, by Theorem 9, the optimal dual vector of the LP relaxation will give us the desired .The steps of this algorithm can be stated as follows.
Algorithm 24 (algorithm to obtain certificate of optimality for MILP).
Result (Certificate of Optimality  * ) (1) For all  ∈ , lift cutting plane    ≤   0 using algorithm in [15] to become valid for the set  in Theorem 9.This is possible by Theorem 20.
(2) Add these cuts to the LP relaxation of the (MILP) and solve this LP which will give (3) Find the optimal dual vector of the LP (, ). is the portion of the dual vector corresponding to the original constraints in (MILP). is a certificate of optimality and   () =  * MILP .
In [16], it is shown that if we have a branch and bound tree instead of a set of cutting planes, we can still get the cutting planes that we need from the tree.This is done by using two different families of cuts, namely, infeasibility cuts (extracted from an infeasible node of the tree) and disjunctive cuts (extracted from a disjunction of the tree or a branch).It is also shown that these cuts could be lifted inexpensively to become valid for the original MILP.However, in this article, we assume that the cutting planes are already available to the user.This can be achieved by using a solver such as Cplex or Coin-OR Cgl (Cut Generating Library).We have used Cgl in some of our computational experiments.

Computational Studies and Numerical Results
When we allow entries of the original matrix  to be any number, we have to include in  all columns with at least one negative element (note that, in case of knapsack problems, a column has only one element).This will increase the size of  compared to the case when all entries of  are nonnegative.Hence, in the case when there are negative entries, we only report the ratio of the number of nonnegative columns in , that is,  0-1 knapsack problems are probably the simplest problems that one could find a certificate for since  has length one.Our computational experiments show that, in most 0-1 mixed integer knapsack problems, the size of  + is significantly smaller than  + .In lower dimensions, the size of  + is usually about 10% of the size of  + , but when | + | is large this ratio will decrease to 1% on average and even less depending on the problem type.In the best case, we had Also, the results are even better when we are working with nonnegative entries, that is, when  and  in (MILP) are nonnegative.This is obvious because all columns with at least one negative element should be put in .However, if we have a problem with lots of negative entries, we can multiply rows of  by −1 to get a better structure of the problem.
All the instances that we work with are generated randomly; that is, the coefficients, objective values, and the right hand side are chosen randomly from specific intervals.These instances include problems with coefficients in {0, 1} and also instances where coefficients vary as the size of the problem increases.
In this section, we represent our numerical experiments for each family of problems.Coin-OR Cbc [17] has been used as MILP solver.Solving times have been only reported for the problems for which it takes Cbc more than 0.1 seconds to solve.All computations have been performed on a 64-bit Intel Core i3 M380 quad core 2.53 GHz CPU with 4 GB of RAM.
Tables 1 and 2 show the results for pure and mixed integer knapsack problems with nonnegative coefficients, respectively.In Tables 3 and 4, we show the results for pure and mixed integer knapsack problems, respectively, without any restriction in input data.
All the instances used for computational experiments are available in MPS or LP file format from http://web.cs.ucdavis .edu/∼bmoazzez/P1.html along with the generated optimal certificates.
For pure integer knapsack problems with nonnegative coefficients, we get the best results.We have several examples with 100,000 variables with ||/|| about 0.02.
We have also done experiments on set covering problems of the form min  s.t. ≥ 1  ∈ {0, 1}  , (45) where 1 is a vector of ones and elements of  are in {0, 1}.The results for set covering problems are very satisfying.On average, we are able to generate good certificates.See Table 5 for set covering certificates.
Our results show that our method successfully generates certificates of optimality for any MILP.In the case of knapsack and set covering problems, our certificates are very good certificates in the sense that they take very short time to verify the optimality compared to the time they take to solve the original problem.Also, we have observed that as the number of variables increases we can generate better certificates.The size of the best certificate that we have generated is roughly 1% of the size of the original MILP, and it takes about 10 milliseconds to verify the optimality while it takes 29970 milliseconds to solve the original problem which means that it is about 3000 times faster to verify the optimality.

Summary and Conclusion
We have generalized the definition of subadditive generator functions and subadditive ray generator functions to mixed integer programming problems without any restriction in the input data.We have shown that this family of functions is very strong in the sense that one can achieve strong duality considering this class solely in the subadditive dual.Also, it is shown that the convex hull of a MILP could be represented using a finite number of these functions.Our main result states that these functions can be used as a certificate of optimality for specific classes of MILP problems.Checking certificates generated using this method is computationally less expensive.Our computational experiments show that for the knapsack problem (pure or mixed) the ratio of  over  in the best case is 1 percent.
last inequality holds because  ≥ 0 and ∑ ∈    is a valid inequality for the set { : ∑ ∈     ≤ ,  ≥ 0,   ∈ Z∀ ∈ } (which follows from the fact that    ≤   0 is valid for  and  ≥ 0 and that  contains all columns which are not entirely nonnegative).Recall here that  = { :  ≥ 0} and  −  belong to  (this implies that  ≤ ).So we get max {∑ ∈(  −   )   : ∑∈     ≤ ,  ≥ 0,   ∈ Z ∀ ∈  ∩ } ≤  −  * MILP .(20)Thisimplies,, () ≥  * MILP .Also, we know that  ,, () ≤  * MILP since  ,, is feasible to subadditive dual problem for (MILP).So  ,, () =  * MILP .Moreover, by Lemma 8,  ,, (0) = 0.If (MILP) is infeasible, with the same  and  as above, Definition 15.A certificate of optimality for a MILP is the information that can be used to check optimality without having to solve the MILP itself.Ideally, we are interested in types of certificates that allow us to perform the checking in (much) shorter time.By Theorem 9, any subadditive generator function with  ,, () =  * MILP for which  is smaller than  can be used as a certificate of optimality.Definition 16.For (MILP) with optimal solution  * ,  is a certificate of optimality if  ,, () =  * .One calls  ∈ R  a "good" certificate if || ≪ . * is called minimal if one has  * = argmin Then,  is a minimal certificate of optimality for (MILP) if and only if the optimal solution of the linear programming relaxation of (MILP) solves (MILP).

Table 1 :
Pure integer knapsack problems with nonnegative coefficients.

Table 2 :
Mixed integer knapsack problems with nonnegative coefficients.

Table 5 :
Set covering problems.