Multiobjective Optimization Involving Quadratic Functions

,


Introduction
Life is about making decisions and the choice of the optimal solutions is not an exclusive subject for scientists, engineers, and economists.Decision making is present in day-to-day life.Looking for an enjoyable vacancy, everyone will formulate an optimization problem to a travel agent, a problem like with a minimum amount of money visit a maximum number of places in a minimum amount of time and with the maximum level of comfort.Usually all real design problems have more than one objective; namely, they are multiobjective.Moreover, the design objectives are often antagonistic.
Edgeworth [1] was the pioneer to define an optimum for multicriteria economic decision making problem, at King's College, London.It was about the multiutility problem within the context of two consumers,  and ."It is required to find a point (, ) such that in whatever direction we take an infinitely small step,  and  do not increase together but that, while one increases, the other decreases." Few years later, in 1896, Pareto [2], at the University of Lausanne, Switzerland, formulated his two main theories, Circulation of the Elites and the Pareto optimum."The optimum allocation of the resources of a society is not attained so long as it is possible to make at least one individual better off in his own estimation while keeping others as well off as before in their own estimation." Since then, many researchers have been dedicated to developing methods to solve this kind of problem.Interestingly, solutions for problems with multiple objectives, also called multicriteria optimization or vector optimization, are treated as Pareto optimal solutions or Pareto front, although, as Stadler [3] observed, they should be treated as Edgeworth-Pareto solutions.
Extensive reviews of multiobjective optimization concepts and methods are given by Miettinen [4], for evolutionary algorithms by Goldberg [5] and for evolutionary multiobjective optimization by Deb [6].The theoretical basis for multiobjective optimization adopted in this work was based on these references.
Thanks to the computer development, optimization of large scale problems became a common task in engineering designs.The development of high speed computers and their increasing use in several industrial branches led to significant changes in the design processes.Currently, the computers, each time faster, allow the engineer to consider a wider range of design possibilities and optimization processes allow systematic choice between alternatives, since they are based on some rational criteria.If used adequately, these procedures can, in most cases, improve or even generate the final results of a design.
Associated with computer development, many of the research done in optimization is focused on numerical methods to solve any kind of problem, but sometimes simplified problems can give important clues to the designer during the trade-off phases of a decision.
The present work aims to bring new approaches to solve multiobjective optimization problems, providing a rapid solution for the Pareto set if the objective functions involved are quadratic.
The rest of the paper is organized into 3 sections.In the first section a general multiobjective optimization problem is formulated and the nature of optimal solutions from the Pareto perspective and the necessary conditions to be met are defined.In the second section, three propositions are done to solve the unconstrained multiobjective optimization problems involving quadratic functions.In the first section the general problem comprises two bidimensional functions.In this case, the proposition permits to find the Pareto front analyticaly.In the second section, the problem considers the minimization problem with three or more functions, keeping the decision space in two dimensions.In this case the proposition helps to find the Pareto points and their boundary in the decision space.In the third section, proposition of the decision space is expanded to any dimensional size.Finally, a section with the conclusions and the proposed future work is presented.

Multiobjective Optimization Problem
Multiobjective optimization problems (MOOP) can be defined by the following equations: where f(X) = [ 1 ,  2 ,  3 , . . .,   ]  : R  → R  is a vector with the values of scalar objective functions   (X) : R  → R to be minimized.X ∈ R  is the vector containing the design variables, also called decision variables, defined in design space R  .X inf and X sup are, respectively, the lower and upper bounds of the design variables.  (X) : R  → R represents the th inequality constraint function and ℎ  (X) : R  → R the th equality constraint function.Equations (1b) to (1d) define the region of feasible solutions, S, in design space R  .The constraints   (X) are of type "  () ≤ 0" functions in view of the fact that "  () ≥ 0" functions may be converted to the first type if they are multiplied by −1.Similarly, the problem considers the "minimization" of   (X), given the fact that function "maximization" can be transformed into the former by multiplying it by −1.

Pareto Optimal Solution.
The notion of "optimum" in solving problems of multiobjective optimization is known as "Pareto optimal." A solution is said to be Pareto optimal if there is no way to improve one objective without worsening at least another; that is, the feasible point X * ∈ S is Pareto optimal if there is no other feasible point X ∈ S such that for all  <>    (X) ≤   (X * ) and   (X) <   (X * ).
Due to the conflicting nature of the objective functions, the Pareto optimal solutions are usually scattered in the region S, a consequence of not being able to minimize all the objective functions simultaneously.In solving the optimization problem we obtain the Pareto set or the Pareto optimal solutions defined in the design space and the Pareto front, an image of the objective functions, in the criterion space, calculated over the set of optimal solutions.

Necessary Condition for Pareto Optimality.
In fact, optimizing multiobjective problems expressed by (1a)-(1d) is of general character.The equations represent the problem of single-objective optimization when  = 1.According to Miettinen [4], such as in single-objective optimization problems, the solution X * ∈ S for the Pareto optimality must satisfy the Karush-Kuhn-Tucker (KKT) condition, expressed as follows: where   is the weighting factor for the gradient of the th objective function, calculated at the point X * , ∇  (X * ).  represents the weighting factor for the gradient of the th inequality constraint function, ∇  (X * ), and is zero when the constraint function associated is not active; that is,   (X * ) ≤ 0.   represents the weighting factor for the gradient of the th equality constraint function, ∇ℎ  (X * ).Equations (2a) to (2e) form the necessary conditions for X * to be a Pareto optimal as described by Miettinen [4].They are sufficient for complete mapping of the Pareto front if the problem is convex and the objective functions are continuously differentiable in the S space.Otherwise, the solution will depend on additional conditions, as shown by Marler and Arora [7].
The methods we will propose in the next sections can be classified in posteriori preference articulation and an extensive literature review of the most important methods to solve multiobjective optimization problems can be found in Augusto et al. [8].

Two-Dimensional Functions of Class C 1
In this section we propose a simple strategy to determine the Pareto set in the decision space and the corresponding Pareto front in the function space, for MOOP involving two bidimensional differentiable functions.Consider an unconstrained multiobjective optimization problem.From (2a), the optimality condition can be interpreted by the following proposition.
Proposition 1.If there exists a Pareto front for the minimization problem with two continuous and differentiable functions defined in R 2 , say  1 ( 1 ,  2 ) and  2 ( 1 ,  2 ), then the points in the decision space, where the gradients of both functions are parallel and opposite, define a continuous Pareto set that connects both functions minima.
As the gradients of each function are orthogonal to contours and point outwards from the minimum, the curve mentioned in Proposition 1 is the locus where the gradients of both functions are parallel and opposite, as shown in Figure 1.
3.1.Two Quadratic Functions Defined in R 2 Space.Proposition 1 is quite general, but as our focus is on quadratic functions let us solve an unconstrained biobjective optimization problem involving quadratic functions defined in the twodimensional decision space; that is, The problem is defined as follows: minimize: Applying the optimality condition, ∑  =1   ∇  (X * ) = 0, to (3a) and (3b) results in the following: As the system of ( 4) is homogeneous, the nontrivial solution, with  ̸ = 0, requires a singularity; that is, the determinant of the coefficient matrix must be null.Consider which results in the following quadratic curve for ( 1 ,  2 ): where Function gradients ∇ 1 (X) and ∇ 2 (X) are parallel on the curve defined by ( 6), but they have to be opposite, resulting in positive weights in (4).Being the system singular, to find a relation between the weights  1 and  2 we can use only one of the equations as the other is its linear combination.Using the first equation, this relation can be deduced as follows: which have positive values if and only if Therefore, (6) provides the locus where the functions gradients are parallel and (9) defines the Pareto set for the two quadratic functions minimization problem.The upper bound of ( 9) As both terms are the first components of ∇ 1 and ∇ 2 , respectively, these conditions imply that the solution ( * 1 ,  * 2 ) is over  1 ( 1 ,  2 ) minimum or over  2 ( 1 ,  2 ) minimum.In conclusion, the Pareto set for quadratic functions will be a quadratic curve connecting the functions minima and where the gradients are parallel and opposite.As an example, let us consider the following biobjective problem: minimize: From ( 6), the Pareto set takes the form and is constrained by the following inequality: In Figure 2(a) the contours of functions  1 and  2 in the two-dimensional decision space are depicted.The thicker grey continuous curve represents (12) and the thick blue coloured portion of this curve satisfies (13), being as expected the continuous Pareto set, namely, the curve along which the gradient vectors are parallel and opposite.In Figure 2(b), the continuous curve is the image of the Pareto set in the function space, that is, the Pareto front.In addition, the blue dots points are the images of the optimization functions calculated on a regular grid in the design space.
For comparison, it is shown in Figures 2(c) and 2(d), adapted from Augusto et al. [8], that the solution was obtained by the genetic algorithm NSGA II of Deb et al. [9].It can be seen that the points are evenly distributed in the function space, but they are not in the decision space.
That happens because the search procedure in most of GAs is focused in the function space, trying to get a well-distributed Pareto front.

Three or More Functions Defined in R 2
Space.In the previous section we found a closed form solution for the optimization of two quadratic functions in the bidimensional decision space.Unfortunately, we did not find a similar solution when we add more functions in the problem.Nevertheless, the idea behind Proposition 1 remains useful.
An equilibrium condition exists when ∇ 1 (p) is oriented through the opposite angular sector defined by the two other gradient vectors, namely, ∇ 2 (p) and ∇ 3 (p).
Based on this idea we suggest the following.
Proposition 2. Let e  be the unit vector defined by e  = ∇  (p)/‖∇  (p)‖, with ‖∇  (p)‖ ̸ = 0, and e  , the unit vector orthogonal to e  ; that is, e  ⋅ e  = 0.If p belongs to the Pareto set resulting from the multiobjective optimization problem involving continuous and differentiable functions defined in R 2 , then there exist at least three unit vectors, say, e  (p), e  (p), and e  (p), that satisfy the following conditions: (e  ⋅ e  ) < 0, (15b) The direction of e  divides the decision space in two semiplanes.If the vector ∇  (p) points to one side, then (15a) and (15b) state that the vectors ∇  (p) and ∇  (p) point to the other side and (15c) states that −∇  (p) is placed between them.
Equations (15a), (15b), and (15c) form a condition test for a point be Pareto optimum or not.This test can be useful if the problem has few optimization functions as to explore all distinguished sets with three gradient vectors in a problem with  objective functions; the maximum of !/(−3)!permutations of , ,  must be checked.Let us apply Proposition 2 to find the solution of an unconstrained MOOP with three quadratic objective functions with two of them being those defined by (11a) and (11b) and the third defined by In Figure 4, the Pareto set found applying the Pareto test in the points of a regular grid in the design space divided in 3.3.Quadratic Functions Defined in R  Space.In the former two sections we have considered unconstrained MOOP with quadratic functions defined in the two-dimensional space.
To proceed to larger dimensions, let us define a quadratic function in the R  space, (X) : R  → R, written as follows: with and X  ∈ R  is a local coordinate system for a convenient definition of (X), X 0 ∈ R  is the position of the local coordinate system related to the global one, and T is the coordinates transformation matrix, from local to global coordinates systems.Using ( 18), ( 17) can be rewritten as follows: Calling A r = (T  AT), (19) can be rewritten as follows: As (X) is smooth, its gradient vector is Matrix A, as well as its transformed form A r , is the symmetric Hessian of (X), H(X), containing its second partial derivatives.With these definitions, let X * be the solution of an unconstrained MOOP involving  quadratic functions defined in R  space.Accordingly, there exists   ≥ 0,  = 1, . . ., , that satisfy (2a); that is, As   (X) is a quadratic, (21) can be used and ( 22) takes the form In ( 23), the weights   , as well as the searched solution X * , are unknown.Let us assume that all   are known; that is,   =  *  .Accordingly, (23) can be rewritten as follows: Calling then (24) can be rewritten as follows: Let us assume that all A r are positive definite; that is, is real, nonnegative and satisfies the normalization equality, that is, ∑  =1  *  = 1, then Â will also be positive definite and therefore its inverse Â−1 will always exist.
Consequently, the Pareto optimum solution X * can easily be found by solving (27); that is, In this approach, we have considered that  *  are known.Consequently, Â, (25), and b, (26), are promptly found.Although this is not the case for a general solution of (22), this approach is very useful to find the Pareto set and the Pareto front of unconstrained multiobjective optimization problems involving quadratic functions considering the following.(5) Repeat steps ( 1) to ( 4) for the number   of Pareto points wanted.
Even requiring solutions of   linear systems, the method is very fast depending on the order of the matrix Â.
Before advancing to the applications, consider an ellipsoid enclosed in a parallelepiped of sizes 2, 2, and 2, as shown in Figure 5. Also consider local coordinates system, X  = [ 1 ,  2 ,  3 ]  with origin centered inside the ellipsoid, fixed to it, and oriented along its semiaxes.
The family of quadratic functions that represents this ellipsoid can be written as follows: with the matrix A defined in Figure 5.
The ellipsoid can be rotated around the th coordinate axis; that is, X r  = r  X  .Let , , and  be the rotation angles, around  1 ,  2 , and  3 axes, respectively.Each individual rotation matrix is depicted in Figures 9(a), 9(b), and 9(c) and the appendix.Then, the general rotation matrix is defined by The local coordinate system can be positioned at a point X 0 , relative to a global coordinates system, X = [ 1 ,  2 ,  3 ]  .In such a case, the points on the surface of the ellipsoid can be referenced in the global system as To get the transformation matrix T of (18), we isolate X  in (31); that is, Being R an orthogonal matrix, its inverse is equal to its transpose; that is, With the previous definitions, consider the following unconstrained MOOP: minimize:  1 (X) ,  2 (X) ,  3 (X) with X ∈ R 3 , with  1 (X),  2 (X),  3 (X) defined in Table 1 and illustrated in Figure 6(a).3 and the appendix.
The Pareto set for this problem, illustrated in Figure 6(b), was obtained by applying Proposition 3 algorithm, with   = 5000.To get all the points, an ordinary 2 GHz dual processor computer with 3 Gb RAM, running Matlab, expended 0.99 seconds of processing time.
As all ellipsoids were placed over ( 1 ,  2 ) plane and were rotated around  3 axis, only, the Pareto set is over the ( 1 ,  2 ) plane.Bold points at the Pareto set boundary were found with the same method applied to the functions  1 (X),  2 (X),  3 (X) taken in pairs.According to Proposition 1, in such cases, the Pareto set is necessarily a curve.The Pareto front is shown in Figure 6(d).It should be noticed that this front was obtained by means of a straightforward solution of the Pareto optimality conditions without using any iterative algorithm.
In the next example three ellipsoids with different orientations, as defined in Table 2 and the appendix, were distributed in the ( 1 ,  2 ,  3 ) space.V e c t o ro f  ∇: Gradient operator.

∇f 1 Figure 1 :
Figure 1: Graphical representation of Proposition 1.The continuous Pareto set as the locus, where objective function gradients are parallel and opposite.

Figure 2 :
Figure 2: Graphical representation of Proposition 1.The continuous Pareto set as the locus, where objective function gradients are parallel and opposite.

Figure 3 :
Figure 3: Pareto optimality condition for three or more functions in R 2 decision space.

Figure 4 :
Figure 4: Pareto optimality condition applied to the three-objective optimization problem involving functions defined in the twodimensional decision space.

Proposition 3 .
Consider a MOOP involving  quadratic functions, with the Hessian of each function being positive definite.To obtain   Pareto optimum solutions the following steps are proposed.

Figure 5 :
Figure 5: Representation of an ellipsoid, a quadratic function (X) = 0 defined in R 3 space.

Figure 6 :
Figure6: Solution of the unconstrained MOOP with the quadratic functions defined in Table1.

Figure 7 :
Figure 7: Solution of the unconstrained MOOP with the quadratic functions defined in Table2and the appendix.

3 Figure 8 :
Figure 8: Pareto set of the unconstrained MOOP with the quadratic functions defined in Table3and the appendix.

Table 1 :
Coefficients for objective functions   (X) definitions.

Table 2 :
Optimization problem with 3 objective functions.

Table 3 :
Optimization problem with 4 objective functions.