Fast Polynomial Time Approximate Solution for 0-1 Knapsack Problem

0-1 Knapsack problem (KP) is NP-hard. Approximate solution is vital for solving KP exactly. In this paper, a fast polynomial time approximate solution (FPTAS) is proposed for KP. FPTAS is a local search algorithm. The best approximate solution to KP can be found in the neighborhood of the solution of upper bound for exact k-item knapsack problem (E-kKP) where k is near to the critical item s. FPTAS, in practice, often achieves high accuracy with high speed in solving KP. The computational experiments show that the approximate algorithm for KP is valid.


Introduction
0-1 Knapsack problem (KP) is a classical combinatorial optimization problem. KP is NP-hard. Knapsack problems were used to model capital budgeting problems in which investment projects are to be selected subject to expenditure limitations. Additionally, the knapsack problem has been used to model loading problems. Knapsack problems, moreover, arise as suboptimization problems in solving larger optimization problem [1]. KP can be formulated as follows: x i � 0, 1.i � 1, 2, · · · , n, (1) where p i is the profit of item i, w i is the weight of item i, c is the capacity of the knapsack, n is the number of items in KP, and variable x i � 0 or 1 indicates whether item i is selected or not. Without loss of generality, it is assumed that all items are arranged in nonincreasing order of efficiency.
It is very important to find a fast polynomial approximation for KP in practice. Greedy algorithm [1], an approximate algorithm for KP, is on the basis of certain rules, such as the item being selected with priority over larger efficiency, larger profit, or smaller weight. Greedy algorithm is O(n) time complexity, but it is not an ε-approximate algorithm [1]. Approximation algorithm is often k-neighborhood local search method. Increasing the radius k of the neighborhood can improve the accuracy of the approximate algorithm [1,2]. While PTAS for KP typically require only O(n) storage, all FPTAS are based on dynamic programming and their memory requirement increases rapidly with the accuracy ε, which makes them impractical even for relatively big values of ε [3]. Heuristics rules are adopted to decrease the calculation in searching accurate approximation, such as harmony search algorithm [4,5], amoeboid organism algorithm [6], cuckoo search algorithm [5,7], binary monarch butterfly optimization [8], cognitive discrete gravitational search algorithm [9], bat algorithm [10], and wind driven optimization [11]. Nowadays, it is tend to combine different heuristics together in solving combinatorial optimization problem, such as mixed-variable differentiate evolution [12], self-adaptive differential evolution algorithm [13], two-stage cooperative evolutionary algorithm [14], cooperative water wave optimization algorithm with reinforcement learning [15], and cooperative multi-stage hyper-heuristic algorithm [16]. However, these algorithms cannot guarantee the accuracy of the solution of KP. Meanwhile, these methods for the solution to KP are generally time-consuming. More domain knowledge is necessary for better algorithm. Pisinger gave an exact algorithm for KP [17] which is based on an expanding core. e items not included in the core are certain to be selected or not in the optimal solution, while the items in the core are uncertain to be selected or not in the optimal solution. He found that algorithms solving some kinds of core problem may be stuck by difficult cores [18]. For example, KP is determined by It is easy to find that the core of KP is [1,10000], so this problem is difficult to tackle by the exact algorithm in [17]. In this paper, it is easy to get an approximate solution of KP which objective value is 6395122580. e solution can be proved to be the optimal solution.
In this paper, a fast polynomial approximate solution is proposed for 0-1 knapsack problems based on the solution of its upper bound. Firstly, an upper bound is presented based on the exact k-item knapsack problem E-kKP in Section 2. Secondly, an initial solution of KP is constructed on the basis of the solution of upper bound of E-kKP in Section 3.1. irdly, the approximate solution is proposed to find the best solution in the neighborhood of the initial solution in Section 3.2. In Section 3.3, the best approximation solution is achieved by comparison with number of items changing. In Section 3.4, the calculation for the approximate solution of KP is analyzed. In Section 4, computational experiments of KP are implemented. e results show that the approximate solution proposed in this paper can achieve high accuracy in general. It implicates that the exact solution to KP is similar to the solution to the upper bound. e algorithm proposed here is a fast polynomial approximate solution to KP.

Upper Bound for KP
If there are exact k objects selected in the knapsack, then KP is an exact k-item knapsack problem E-kKP formulated as follows [3]: (4) e upper bound of E-kKP may be achieved by Lagrangian relaxation of capacity constraint: Suppose that So L(k, λ) can be solved by sorting p i − λw i in descending order in O (nlnn) time if λ and k are fixed. e upper bound of E-kKP is formulated as follows: It can be proved that L(k, λ) is a unimodal function of λ if k is fixed. B k can be quickly solved by linear search algorithm [19]. So B k can be solved in O (nlnn) time.
e upper bound of KP is the maximum value of B k : It can be proved that B k increases when k < s − 1 and decreases when k > s, where the critical item s satisfies.
So the upper bound of KP is 486.9565. We test different upper bounds of KP from Pisinger's paper [20], where the weights and profits of items are randomized. e instances listed in Table2 are tested to compare our proposed method with the existing upper bounds, such as the upper bound U MT2 proposed by Martello and Toth [21], the improved upper bound U MTM [22] and the upper bound U kmax with maximum cardinality [23]. e difficult instances can be constructed as follows [20]: e weights are uniformly distributed in [1, R] and for each weight w the corresponding profit is chosen as where d is 2/3. (ix) Profit ceiling instances pceil (d): e weights of the n items are randomly distributed in [1, R], and the profits are set to p i � d w i /d . e parameter d was chosen as d = 3.
(x) Multiple strongly correlated instances mstr (k 1 , k 2 , d): e weights of the n items are randomly distributed in [1, R]. If the weight w i is divisible by d, then we set the profit p i : = w i + k 1 ; otherwise set it to p i : = w i + k 2 . We set d: = 6 here.
For each instance type, a series of K � 100 instances is performed, and the capacity is determined by (13). All the instances above are generated with data range R � 10 3 , 10 4 , 10 5 , 10 6 or 10 7 .
ere are 100 instances for each type of KP where the capability is described as follows: e mean relative error of upper bounds to the best upper bound of KP is listed in Table 3.
From Table 3, we find that the relative error of the upper bound B is the minimum in general. e upper bounds of 5000 instances are carried out with different methods. More details are listed in Table 4.
From Table 4, we find that U kmax is the best upper bound in 2025 instances. e upper bound B plays an important role in obtaining the best upper bound even if s ≤ k max . e upper bound can gather most items selected in the optimal solution all together. For example, KP is determined by A maximum of 631 items can be selected in the knapsack. e upper bound B s− 1 (s � 632) and the solution u (s− 1) are as follows: Computational Intelligence and Neuroscience e initial solution z (s− 1) is equal to u (s− 1) . e best solution y (s− 1) a in the neighborhood ∪ 4 h�1 N(z (s− 1) , 2h) can be described as follows: It is obviously that there is little difference between z (s− 1) and y (s− 1) a . Relation between (p i − 106310w i ) and w i is displayed in Figure 1. Relation between p i /w i and w i is displayed in Figure 2. It is found that (p i − 106310w i ) changes more dramatically than p i /w i . It makes solution to KP easy.
From Figure 1, we can see that (p i − 106310w i ) of all items selected are larger, while the others are smaller. It is easy to get a better solution on the basis of (p i − 106310w i ) than that on the basis of efficiency p i /w i .

Approximate Solutions
e approximate solution to E-kKP is a solution to KP. We obtain the best approximate solution to KP by comparison to approximate solutions for E-kKP where k is near to the critical item s. In order to achieve a better solution to E-kKP, we firstly obtain an initial solution on the basis of the solution to the upper bound of E-kKP. en the initial solution is developed by local search in the neighborhood of the initial solution. At last, the approximate solution to KP is the best approximate solution to E-kKP with various k. e upper bound of E-kKP is used to decrease calculation. e solution to the upper bound of E-kKP and the upper bound make key contribution in FPTAS to KP.

Initial Solution to E-kKP.
Let u (k) � (u d ki ) be an optimal solution to the upper bound B k . We may obtain an initial solution to E-kKP on the basis of u (k) . e initial solution z (k) to E-kKP is determined by In Example 1, z (19) is an initial solution and its objective value is

Approximate Solution to E-kKP. Approximate solution to E-kKP is the best solution in the neighborhood of z (k) .
Let N(z (k) , 2h) be a neighborhood of z (k) that is defined by It is obvious that the size of N(z (k) , 2h) increases with h. But it is unnecessary to take into account all elements of N(z (k) , 2h). Algorithm 1 is a fast algorithm for searching the best solution in N(z (k) , 2h).
e best approximate solution corresponds to max p s2 − p s1 | w s2 ≤ w s1 , so the calculation is O (nlnn) time when h equals to 1. A better solution is generated if the profit sum of h items selected in the knapsack is less than that of h items not selected in the knapsack, and the capacity constraint still holds at the same time. On the other hand, the calculation decreases via variable reduction in practice.
In Example 1, let the approximate solution x a equal to z (19) and objective value f a equal to f(z (19) ) firstly, and then the approximate solution x a is updated by the best solution y (19) a in N(z (19) , 2) if possible.

(24)
Similarly, x a is replaced by the best solution y (19) a in N(z (19)  (25) ere is no better solution of KP in N(z (19) , 6) and N(z (19) , 8). We get an approximate solution y (19) a with 19 items selected and the approximate objective value is 486.9426.

Approximation Algorithm of KP.
Let x a describe the best approximate solution to KP in (26).
Step 2: Calculate the objective value of feasible solution y (k) � (y d ki ) in N(z (k) , 2h) by equation as follows: Step 3: Obtain the best solution y (k) ALGORITHM 1: Approximate solution to (E-kKP).

Computational Intelligence and Neuroscience
Step 1 Input the capacity c, size n, weights w i and profits p i of items.
Step 2 Obtain the critical item s by (10).
Step 4 Obtain the upper bound B k and its solution u (k) (k � k 1 , k 2 ) by (8).
Step 6 If B k ≤ f a , then exit.
Step 7 Obtain the initial solution z (k) by (18).
Step 9 If f a < f(y (k) a ), then f a ←f(y (k) a ), x a ←y (k) a .
In Example 1, we obtain f a � 486.9426 when k � 19.
e best approximate solution x a � (x d 19i ) for KP and its objective value are as follows: It may be proved that x a is the optimal solution to KP in Example 1.

Calculation Analysis.
Approximation algorithms for KP based on upper bound of E-kKP runs in polynomial time. Calculation for the upper bound of E-kKP is less than O (nlnn) for a given E-kKP. We obtain the (p e j /w e j ) ≥ (p e j+1 /w e j+1 ), j, l � 1, 2, · · · , 29. (x ei i ) is an initial solution constructed in descending order of efficiency. approximate solution to KP on the basis of the upper bound of E-kKP where k is an integer close to s. For a given k, the initial solution is determined by (13). It takes O (nlnn) time to develop the initial solution in the neighborhood N(z (k) , h) (h = 1, 2, . . ., 4) defined by (21). Hence, it takes at most O (nlnn) time to achieve the approximate solution to E-kKP. Generally, we obtain the best approximate solution for KP among the approximate solutions for E-kKP where k is near the critical item s. Hence, the calculation for approximate solution to KP is less than O (nlnn).
In Algorithm 1, we search for the best solution in N(z (k) , h) defined by (21). In order to develop the approximate solution quickly, the weight sum of h items is sorted in ascending order firstly, and then the profit sum of h items selected in the knapsack is compared with the profit sum of h items not selected. So the storage needed in Algorithm 1 is O(n) when h is fixed at 1, 2, 3, or 4. In Algorithm 2, we have the best solution to KP by comparison to the best approximate solution to E-kKP where k is an integer close to s. So the storage of the approximate algorithm for KP is O(n).
3.5. Accuracy Analysis. All feasible solutions to KP are in the search scope of approximation algorithm with changing k and h. So approximation algorithm for KP is an ε-approximation algorithm. From one aspect, weights, profits, capacity and size of KP have influence on the accuracy of an approximation algorithm. From another aspect, the search scope of the approximation algorithm has influence on its accuracy as well. Better solution is usually with more calculation. e exact algorithm for KP may be explored on the basis of the branch and bound algorithm here. Intensive research will be carried out in the future.

Computational Experiments
By Algorithm 2, we get the approximate solutions to KP listed in Table 5. e results are listed in Table 6. e optimal value listed in Table 6 is carried out by combo [24].
From Table 6, we find that the approximate solutions are almost the optimal solutions in 18 instances. It implicates that the approximate algorithm proposed here achieves high precision in solving KP.
In Table 7 lists the experimental results of the solutions for KP listed in Table 2 by Algorithm 2. e upper bound listed in Table 4 is carried out by equation (11) in Section 2.
From Table 7, we see that the relative average error of 100 instances is almost less than 0.0001. e experiment result shows that the approximate algorithm proposed here can achieve high accuracy.
From Table 8, we find that the upper bound of E-kKP makes key contribution in FPTAS. Firstly, the initial solution constructed on the basis of the solution to the upper bound of E-kKP is similar with the optimal solution. For example, there are only 4 elements different between the initial solution and the optimal solution to E-kKP where k equals to 19, while there are 8 elements different between the initial solution constructed by efficiency and the optimal solution to KP in Table 8. Secondly, the differences between the initial solution and the solution to the upper bound of E-kKP are near to the item d kk . It is to say, we search the optimal solution to KP in a core with small size on the basis of the solution to the upper bound of E-kKP. While the optimal solution to KP in a core with large size on the basis of the solution to the upper bound of Dantzig. So, algorithms proposed here is easy to get approximate solution to KP.
Furthermore, no better solution of E-kKP exists when the objective value f a of the approximate solution achieved before is larger than the upper bound B k . So we only search solution of E-kKP with upper bound larger than f a which is in the neighborhood of the optimal solution to the upper bound of E-kKP. e candidate strategy makes the search in polynomial time and the solution with high accuracy in Algorithm 1. e upper bound of E-kKP plays important role in decreasing calculation of the approximate solution of E-kKP in Algorithm 2 as well.

Conclusion
It is still difficult to obtain the exact solution for large scale 0-1 knapsack problem directly. Here a fast polynomial approximate solution is proposed on the basis of the upper bound for KP. e exact solution to KP is in the neighborhood of the solution to the upper bound for E-kKP. erefore, it is possible to find an approximation with high accuracy in the neighborhood of the solution to the upper bound for E-kKP where k is near to the critical item s. All in all, as the basis of fast exact algorithm for KP, it is important to obtain an approximate solution and the upper bound for KP. In order to obtain a fast exact solution to KP, more intensive research on variables reduction need be conducted in the future.

Data Availability
All data inside the manuscript have been specified clearly in the manuscript.