Dynamic Optimization Technique for Distribution of Goods with Stochastic Shortages

This work considers the distribution of goods with stochastic shortages from factories to stores. It is assumed that in the process of shipping the goods to various stores, some proportion of the goods will be damaged (which will lead to shortage of goods in transit).The cost of the damaged goods is added to the cost of the shipment. A proportion of the total expected cost of the shortage goods is assumed to be recovered and should be deducted from the total cost of the shipment. In order to determine the minimum transportation costs for the operation, we adopt dynamic optimization principles. The optimal transportation cost and optimal control policies of shipping the goods from factories to stores were obtained. We find that the optimal costs of the goods recovered could be determined. It was further found that the optimum costs of distributing the goods with minimum and maximum error bounds coincide only at infinity.


Introduction
A distribution company plans to minimize the cost of distributing  kinds of products from  number of factories to  number of stores.For the case of a company distributing a particular product from one factory to all the stores (a single product), that is, at factory  1 , product  is distributed to all the  stores and factory  2 produce  products and are distributed to all the  stores, and so on; see Nwozo and Nkeki [1].It is also expected that the goods that leave the factories will not come back to the factories (in the case of defective, damaged, etc., items).The company considered  number of control policies  = { 1 ,  2 , . . .,   } to determine which of them will yield the optimum control policy.They also estimated that certain percentage of the products are to reach their final destination successfully at minimum costs.It is assumed that some proportion of the shortage goods should be recovered.The recovered costs of the shortage goods should be deducted from the total costs continuously over time horizon.
In the related literature, Powell [2] considered the problem of a stochastic fleet assets management problem and used the postdecision state variable implicitly to determine this problem.Closely related to this on the extensive work on stochastic fleet assets management problem is the work by Cheung and Powell [3].Godfrey and Powell [4] further used postdecision state variable explicitly to determine similar problem on a stochastic fleet management problem.Van Roy et al. [5] proposed the idea of using a parsimonious sufficient statistics in an application of approximate dynamic programming to inventory management.Mulvey and Vladimirou [6] used the stochastic programming technique in financial asset allocation problems for designing low-risk portfolios, and Mulvey and Ruszczynski [7] designed variety of specialized algorithms that help to solve the problems which involved management of homogeneous resources.They further emphasized that for more realistic problems, resources are heterogeneous and that the resulting optimization problems do not generally exhibit pure network structure.Powell et al. [8] described an algorithm for computing parameter values to fit linear and separable concave approximation to the value function to tackle large-scale problems in transportation and logistics.Furthermore, Powell and Topaloglu [9] described a more complicated variation of the algorithm that improved on execution time and memory requirements and such improvement is often critical for practical application to realistic large-scale problems.They further emphasized that the movement of freight over long distances is subject to random delays.Powell and Van Roy [10] studied and applied approximate dynamic programming to high-dimensional resource allocation problems in area of managing a fleet of trucks.Powell [22] worked extensively on an approximate dynamic programming for large-scale asset management problems for both single and multiple assets.Guestrin et al. [11] used an approximate dynamic programming approach to deal with high-dimensional decision spaces.For now, no other applications of approximate dynamic programming have dealt with such large numbers of decision variables (see, Powell and Van Roy [10]).Spivey and Powell [12] studied the dynamic assignment problem, where a resource can serve only one task at a time.They further proposed a general class of dynamic assignment models that involved adaptive and nonmyopic algorithm.Cogill et al. [13] considered the problem of computing decentralized policies for stochastic systems with finite state and action spaces.They presented an algorithm based on linear programming and used this algorithm to obtain a decentralized policy from a function with special structure that approximates the optimal centralized Q-function.Topaloglu and Kunnumkal [14] proposed two approximate dynamic programming methods to optimize the distribution operations of a company manufacturing a certain product at multiple production plants and ship to different customer location for sale.Hauskrecht and Singliar [15] considered real-world distributions and networks which are unreliable and subject to random failures of its components.They investigated MonteCarlo solutions in stochastic networks in which the expected value of resources allocated before and after the occurrence of stochastic failures needs to be optimized.Ozbay et al. [16] proposed mathematical programming models with probabilistic constraints in order to address incident response and resource allocation problems for traffic incident management.They considered the resource allocation problem by assuming that the stochastic distribution of incidents over a network is given and introduced a mathematical model to determine the number of service vehicles allocated to each depot to meet the requirements of the potential incidents by taking into account the stochastic nature of the resource requirement and incident occurrence probabilities.Jacko [17] considered resource allocation problem in telecommunication system.They optimized packet losses by choosing preferred paths in the routing system.They assumed that there are a number of independent competitors with the aim of maximizing the expected return.Elmaghraby and Ramachandra [18] considered the problem of optimally allocating a single resource under uncertainty to the various activities of a project to optimize a certain economic objective composed of resource utilization cost and tardiness cost.Nwozo and Nkeki [19], used DPP to consider the allocation of buses from single station to different routes in Nigeria for profit maximization.Nkeki [20] considered the allocation of vehicles from different (multiple) stations to different routes putting into consideration the rate of breakdown of the vehicles during the allocation but failed to put into consideration the depreciation of the vehicles and effect tax on the returns of the investment.Nwozo and Nkeki [21] studied the allocation of resource to different tasks in a production company so as to maximize returns.They assumed that the company produces the same kinds of products and allocate  number of tasks to  number of machines.They also assumed that the machines are subject to random breakdown.
This paper discussed the distribution of goods from different factories to stores with stochastic shortage over time horizon.
The highlights of this paper are (a) modeling the distribution of goods from factories to stores in the presence of (i) stochastic shortages of the goods in transit and (ii) deterministic case (i.e., when there are no shortages) was also considered, using a dynamic optimization technique, (b) numerical determination of optimal cost and optimal control policy for the operations was obtained.
The remaining parts of the paper are structured as follows: Section 2 presents the problem formulation and multiperiod expected cost function.In Section 3, we present the dynamic programming formulation of the problem.The optimization of the cost function is determined in this section.Section 4 presents the dynamic programming principles with error bounds.In Section 5, numerical simulations of the problem are discussed.Section 6 concludes the paper.

Problem Formulation
The paper studies a problem where there are  states and  control policies.Let  factory = {  1 ,   2 , . . .,    } be the set of goods produced in the factories and let  store = {  1 ,   2 , . . .,    } be the set of goods that are to be distributed to the stores, where  factory ,  store ∈  and  is the state space of the system.The transition probability matrix corresponding to the control policies  1 ,  2 , . . .,   is given as  (  ) =  , (  ) ,  = 1, 2, . . ., ;  = 1, 2, . . ., . (1) The transition costs are given by Φ , ( subject to where , (   ) ∈  is the set of feasible solutions of problem (4).We can express (4) as the expected discounted minimal cost from period  onward as an optimization over subject to the constraints (5).

Dynamic Programming Formulation
In this section, we formulate the problem using dynamic programming technique.Since   is the state variable at period  and  the state space, we formulate the problem as a dynamic program.Let the cost of the goods that is to be distributed from factory  to store  at period  under policy    be given by  , , (   ), where  , is the transition probability of the goods from factory  to store .Hence, the total expected cost of the goods distributed from factory  to store  at period  > 0 is given by Let  be the proportion of the total expected cost of goods lost in transit (shortage) from factory  to store .Then, (8) becomes , (   ) ,  = 1, 2, . . ., ;  = 1, 2, . . ., . (9) Equation ( 9) represents the total gross shortage cost incurred from the goods lost in transit.
Let   −1 be the cost of distributing the goods from the factories to store  in period  − 1 and    the expected cost of distributing the goods from the factories to store  at period  and let  be the estimated proportion of the goods that are to be recovered from the lost ones, which are expected to go to the stores at period , so that the net shortage cost incurred from the goods lost in transit will be   11) is the transformation equation and is stochastic.We now have the following propositions.
This proposition tells us that the transformation equation ( 11) is deterministic if  = 0 or  = 1.It implies that all the goods distributed from the factories to stores get to their various destination successfully without shortages.Proposition 2. Suppose that there is no recovery of any of the goods lost in transit and  > 0; then  must be zero and (11) becomes The optimal policy can be found by computing the value functions through the optimality equation subject to the constraints (5).Equation ( 14) can be rewritten as follows: where

𝑖,𝑗 𝑡
∈ R  is the quantity of products that is lost in transit from factory  to store  at period  and is a zero vector, it implies that all the quantities of products that left the factories get to the stores successfully without damages or shortage.It can be shown that ( 6) is equal to (14) (see Powell,[22]).We may use ( 6) and ( 14) interchangeably.We now find the best control policy, , which minimizes our problem; that is, we search for We do that by solving the optimality equation   (ii) The operator Υ has a unique fixed point (given  * ).
We now make the following assumptions.
Theorem 5. Let the bounded optimal cost function  :  → R  be -dimensional vectors.Then  satisfies Proof.Let  be a positive integer,   0 ∈  and policy  = { 1 ,  2 , . . .,   }; we can decompose the return into the portion received over the first  stages and over the remaining stages: But Therefore, It therefore follows that (31) and by taking the limit as  → ∞, we obtain Hence, This result shows that our optimization problem converges to a fixed point  * in an infinite horizon.It therefore follows that (34)

Dynamic Programming Principles with Error Bounds
In this section, the optimality equation with error bounds is considered.We established the minimum and maximum error bounds of our problem.We show that at infinity the minimum and maximum error bounds cancelled out.Let It then follows that where Λ   and Λ   are such that Equation ( 37) can be improved upon as follows: For a vector , we compute By subtracting (40) and (41), we have is the cost vector associated with the control policy  and  is the cost per stage vector.It then follows from (37) that for we have Equivalently, for the vector , we have where Now, combining (46) and (47), we have Substituting ( 45) into (50), we have After  steps, we have As  → ∞, we have Let   1 =   /(1 − ); we have Let  = Υ +1  in (54); we have From ( 46), we have that min . Therefore, we have It implies that Similarly,   =‖ Υ − ‖ ∞ .We therefore have that (58) This is the dynamic programming (value iteration) that is bounded above and below.The value (Υ  )(   ) +    , is the value iteration with minimum error bound and (Υ  )(   ) +    is the value iteration with maximum error bound.Now taking the limit as  tends to infinity, we have that ) . (61) The initial estimate of the cost of distributing the goods from factory  to store  in 100000 is given below.(  (  (  (  (  (  (  (  (  (  (  (  ( The company estimates  = 5% of the total goods as shortage in transit and  = 70% of the total expected shortage to be recovered and should be deducted from the total cost of the shipment.The cost of the damaged goods is added to the cost of the shipment.MATLAB was used to solve the problem.The results are presented in the tables.The optimal discounted costs of distributing the goods from factories to stores with shortages are presented in Tables 1 to 7 and Figures 2 and 3, while optimal discounted costs of distributing the goods from factories to stores without shortages are presented in Tables 8 to 10 and Figure 5. Table 1 presents the minimum costs of distributing the goods from various factories to all the stores.Here, error bound is not considered.It was obtained that the minimum costs of distributing the goods from factory 1 to all the eight stores are 5,716,200 naira.The minimum costs of distributing the goods from factory 2 to the eight stores are obtained as 6,034,000 naira, from factory 3 to the eight stores are 6,428,400 naira, and so on.It was observed that the costs of distributing the goods from factory 7 to all the stores yield the highest amount.This may be due to longer distance, nature of the roads, or more shortage costs.
Table 2 presents the optimal costs of distributing goods with minimum error bound from the factories to stores and Table 3 presents the optimal costs of distributing the goods with maximum error bound from the factories to stores.It was observed that the minimum costs of distributing the goods with minimum and maximum error bounds coincide only at infinity.
Table 4 shows the optimal costs of distributing the goods without error bound and where there is no recovery of some damaged goods (i.e., when  = 0).It is observed from Tables 1 and 4 that the costs of distributing the goods will increase when the value of  decreases and vice versa.This is an expected result.Tables 5 and 6 show the optimal costs of distributing the goods with minimum and maximum error bounds, respectively, and where there is no recovery of some damaged goods.The additional costs incurred and the corresponding percentage increase in the additional costs incurred when  = 0 are shown in Table 7.
In Table 7, it was found that the costs of recovered goods in transit from factory 1 to the stores are 228,600 naira with  3.85% when there is no EB and 220000 naira with 3.70% when there is EB from factory 2 to the stores 230,800 with 3.68% when there is no EB, and 240,000 with 3.83% when there is EB and so on.Figure 2 that optimum costs can be attained under policy 1.Hence, the optimum policy for the operation is  1 .We therefore deduce from Figure 2 that  1 <  6 <  4 <  2 <  8 <  3 <  5 <  7 .Figure 3 shows the bar chart representing the minimum costs of distributing the goods from factories to stores by setting  = 99%,  = 0.0%, and  = 5%.This is when there are no goods recovered from the expected lost ones in transit.We also found that policy 1 yields the optimum costs.
Figure 4 shows the relationship between the minimum and maximum error bound.It was observed that at infinity, the minimum and maximum error bounds coincide.We referred to this point as the point of harmony.Comparing the optimal discounted costs of the distribution of goods in Tables 1, 8, 2, 9, 3, and 10 and Figures 2 and 5, we found that the optimal cost is at minimal when there are no shortages, which is an expected result.It is obvious that the higher the shortages, the higher the cost of the shipment.This may lead to the high cost of the goods on the final consumers.

Conclusion
We have considered distribution of goods with stochastic shortages from factories to stores.This paper modeled the distribution of goods from factories to stores in the presence of stochastic shortages of the goods in transit, and the deterministic case (i.e., when there are no shortages) was also considered, using a dynamic optimization technique.Numerical determination of optimal cost and optimal control policy for the operations were obtained.We obtained the optimal transportation cost of shipping goods from factories to stores.The optimal control policies were also obtained.We found that the optimal costs of the goods could be determined.It was found that the optimum costs of distributing the goods with minimum and maximum error bounds coincide only at infinity.We referred to this point as a "point of harmony."
infimum over , we obtain for all

1 Figure 1 :
Figure 1: Transition diagram for the distribution of goods from factories to stores.

Figure 2 :
Figure 2: The bar chart representing the discounted minimum costs of the distribution with 8 different policies for  = 99%,  = 70%, and  = 5%.

Figure 3 :
Figure 3: The bar chart representing the discounted minimum cost of the distribution with 8 different policies for  = 99%,  = 0.0%, and  = 5%.This occurs when there is no recovery of the goods.

Figure 4 :
Figure 4: The maximum and minimum error bound showing the point of their intersection "point of harmony" at  = 99%,  = 70%, and  = 5%.

Figure 2
Figure2shows the bar chart representing the minimum costs of distributing the goods from factories to stores by setting  = 99%,  = 70%, and  = 5%.It was observed from

Figure 5 :
Figure5: The bar chart representing the discounted minimum cost of the distribution with 8 different policies for  = 99%,  = 0.0%, and  = 0.0%.This is when the goods get to their various stores successfully without shortages.
where  is the terminal time period in the planning horizon,  ∈   at period  and the state of goods to store  is , (
ODCDG stands for optimal discounted costs of distributing the goods.

Table 4 :
ODCDG where there is no recovery; that is, when  = 0.

Table 5 :
ODCDG with minimum EB and where there is no recovery.

Table 6 :
ODCDG with maximum EB where there is no recovery.

Table 7 :
Optimal discounted costs of recovered goods.

Table 10 :
ODCDG without shortage with maximum EB.