Download probability distribution and more Lecture notes Production and Operations Management in PDF only on Docsity!
CHAPTER 12
Deterministic Dynamic Programming
Real-Life Application—Optimization of Crosscutting and Log Allocation
aV|WG[GrJaGWsGr
Mature trees are harvested and crosscut into logs to manufacture different end products
(construction lumber, plywood, wafer boards, or paper). Log specifications (e.g., length
and end diameters) differ depending on the mill where the logs are processed. With
harvested trees measuring up to 100 ft in length, the number of crosscut combinations
meeting mill requirements can be large, and the manner in which a tree is disassembled
into logs can affect revenues. The objective is to determine the crosscut combinations
that maximize the total revenue. The study uses dynamic programming to optimize the
process. The proposed system was first implemented in 1978 with an annual increase in
profit of at least $7 million. Details of the case are presented at the end of the chapter.
12.1 RECURSIVE NATURE OF DYNAMIC PROGRAMMING (DP)
COMPUTATIONS
The main idea of DP is to decompose the problem into (more manageable) subprob-
lems. Computations are then carried out recursively where the optimum solution of
one subproblem is used as an input to the next subproblem. The optimum solution
for the entire problem is at hand when the last subproblem is solved. The manner in
which the recursive computations are carried out depends on how the original prob-
lem is decomposed. In particular, the subproblems are normally linked by common
constraints. The feasibility of these common constraints is maintained at all iterations.
Example 12.1-1 (SJQrVGsV-RQWVGPrQblGm)
Suppose that we want to select the shortest highway route between two cities. The network in
Figure 12.1 provides the possible routes between the starting city at node 1 and the destination
city at node 7. The routes pass through intermediate cities designated by nodes 2 to 6.
470 Chapter 12 Deterministic Dynamic Programming
We can solve this problem by enumerating all the routes between nodes 1 and 7 (there
are five such routes). However, exhaustive enumeration is computationally intractable in large
networks.
To solve the problem by DP, first decompose it into stages as delineated by the vertical
dashed lines in Figure 12.2. Next, carry out the computations for each stage separately.
The general idea for determining the shortest route is to compute the shortest (cumulative)
distances to all the terminal nodes of a stage and then use these distances as input data to the
immediately succeeding stage. Starting from node 1, stage 1 reaches three end nodes (2, 3, and 4),
and its computations are simple.
Stage 1 Summary.
Shortest distance from node 1 to node 2 = 7 miles (from node 1)
Shortest distance from node 1 to node 3 = 8 miles (from node 1)
Shortest distance from node 1 to node 4 = 5 miles (from node 1)
Start 3 7
FIGURE 12.
Route network for Example 12.1-
f
1
f
1
f
0
f
2
f
2
f
3
FIGURE 12.
Decomposition of the shortest-route problem into stages
472 Chapter 12 Deterministic Dynamic Programming
Recursive Equation. This section shows how the recursive computations in
Example 12.1-1 can be expressed mathematically. Let f
i
(x
i
) be the shortest distance to
node x
i
at stage i, and define d 1 x
i - 1
, x
i
2 as the distance from node x
i - 1
to node x
i
. The
DP recursive equation is defined as
f
0
1 x
0
= 12 = 0
f
i
1 x
i
2 = min
all feasible
1 x
i - 1
, x
i
2 routes
5 d 1 x
i - 1
, x
i
2 + f
i - 1
1 x
i - 1
2 6, i = 1, 2, 3
All distances are measured from 0 by setting f
0
1 x
0
= 12 = 0. The main recursive
equation expresses the shortest distance f
i
(x
i
) at stage i as a function of the next node, x
i
.
In DP terminology, x
i
is referred to as the state at stage i. The state links successive stages
in a manner that permits making optimal feasible decisions at a future stage indepen-
dently of the decisions already made in all preceding stages.
The definition of the state leads to the following unifying framework for DP.
Principle of Optimality. Future decisions for all future stages constitute an optimal
policy regardless of the policy adopted in all preceding stages.
The implementation of the principle of optimality is evident in the computations
in Example 12.1-1. In stage 3, the recursive computations at node 7 use the shortest
distance to nodes 5 and 6 (i.e., the states of stage 2) without concern about how nodes
5 and 6 are reached from the starting node 1.
The principle of optimality does not address the details of how a subproblem
is optimized. The reason is the generic nature of the subproblem. It can be linear or
nonlinear, and the number of alternative can be finite or infinite. All the principle
of optimality does is “break down” the original problem into more computationally
tractable subproblems.
AJa!MQmGPV:SQlvKPIMarrKaIGPrQblGm...wKVJD[PamKEPrQIrammKPI!
The German mathematician Johannes Kepler (1571–1630), arguably one of the greatest as-
tronomers ever, was faced with a personal problem: He was seeking a compatible spouse and a
stepmother for his young children after his wife died. A marriage broker presented him with 11
candidates, and over a span of two years he interviewed them one at a time. He rejected some
for lack of compatibility but could not make up his mind regarding the remaining ones, who in
turn, tired of waiting, withdrew their names. After much agonized vacillation, he re-wooed the
fifth woman he interviewed, and the union was a happy one.
Kepler’s problem, initially dubbed as the marriage problem and later as the secretary
(selection) problem, generated considerable interest starting in 1960. The solved version posed
additional restrictions that were not followed by Kepler himself: Given a pool of n applicants
seeking to fill a single position, candidates are interviewed one at a time in random order.
Following each interview, an irrevocable decision is made to accept or reject the candidate. Ac-
ceptance of a candidate ends the process; otherwise the next candidate, if any, is interviewed. If
all the first n - 1 candidates have been rejected (or if n = 1 ), then candidate n must be accepted.
Finding the best candidate in the pool is complicated by the irrevocable accept/reject deci-
sion immediately following each interview. Short of interviewing all n candidates (in which case
the absolute best candidate could be determined), a proposed game strategy calls for rejecting
12.2 Forward and Backward Recursion 473
the first r - 1 candidates (r is yet to be determined from the solution) and then continuing the
interview process, stopping at the first applicant who is better than all the ones rejected. This strat-
egy makes use of previous interviewing experiences in the hope of finding a better (possibly the
best) future candidate, and it is more efficient because it could stop short of interviewing all n
candidates. One way to optimize the decision problem is to determine the cutoff r that maximizes
the probability that a future applicant i is better than the first 1 r - 12 rejected candidates.
The described problem (and its variants) was solved by dynamic programming.
1
Other
solution models include probability theory, linear programming, and Markov chains.
2
The solu-
tions show that the desired probability, defined as P 1 r n 2 , is concave in r and that
P 1 r n 2 Ú max 5 lim
n
S ∞
P 1 r n 26 =
1
e
1
≈ .37, for all n 7 1
This remarkable simple result says that for r ≈ .37n + 1 , there is at least a notable 37% chance
that a future candidate i Ú r is better than the first r - 1 candidates, no matter how large n is.
The proposed solution was actually realized in Kepler’s case when he married candidate
number 5 1 note that .37 * 11 = 4.07 2. In all likelihood, however, the outcome is purely coinci-
dental because Kepler did not quite follow the rules of the proposed problem. Not to mention
that, per published accounts, Kepler first tried to woo candidate number 4 but was unsuccessful.
Nevertheless, the conjecture is a story worth telling!
- 2 FoRWARDAnDBAckWARDRecuRSion
Example 12.1-1 uses forward recursion in which the computations proceed from stage 1
to stage 3. The same example can be solved by backward recursion, starting at stage 3 and
ending at stage 1.
Naturally, both the forward and backward recursions yield the same optimum.
Although the forward procedure appears more logical, DP literature mostly uses back-
ward recursion. The reason for this preference is that, in general, backward recursion
can be more efficient computationally.
We will demonstrate the use of backward recursion by applying it to
Example 12.1-1. The demonstration will also provide the opportunity to present the
DP computations in a compact tabular form.
Example 12.2-
The backward recursive equation for Example 12.2-1 is
f
4
1 x
4
f
i
1 x
i
2 = min
all feasible
routes 1 x
i - 1
, x
i
2
5 d 1 x
i - 1
, x
i
2 + f
i - 1
1 x
i - 1
26 , i = 1, 2, 3
The order of computations is f
3
S f
2
S f
1
1
Beckmann, M., “Dynamic Programming and the Secretary Problem,” Computers and Mathematics with
Applications, Vol. 19, No. 11, pp. 25–28, 1990.
2
Thomas S. Ferguson, “Who Solved the Secretary Problem?” Statistical Science, Vol. 4, No. 3, pp. 282–289,
- Stable URL: http://www.jstor.org/stable/2245639 accessed 7-29-2015 9:10 P.M.
12.3 Selected DP Applications 475
Of the three elements, the definition of the state is usually the most subtle. The appli-
cations presented here show that the definition of the state varies depending on the
situation being modeled. Nevertheless, as you investigate each application, you will
find it helpful to consider the following questions:
- What relationships bind the stages together?
- What information is needed to make feasible decisions at the current stage
without regard to how the decisions made at the preceding stages have been
reached?
You can enhance your understanding of the concept of the state by questioning
the validity of the way it is defined here. Try another definition that may appear “more
logical” to you, and use it in the recursive computations. You will soon discover that
the definitions presented here are correct. Meanwhile, the associated mental process
should give you a better understanding of the role of states in the development of DP
recursive equation.
12.3.1 kPapsaEM/Fl[-Awa[kKV/carIQ-LQadKPIMQdGl
The knapsack model classically deals with determining the most valuable items a com-
bat soldier carries in a backpack. The problem represents a general resource allocation
model in which limited resources are used by a number of economic activities. The
objective is to maximize the total return.
3
The (backward) recursive equation is developed for the general problem of al-
locating n items to a knapsack with weight capacity W. Let m
i
be the number of units
of item i in the knapsack, and define r
i
and w
i
as the unit revenue and weight of item i.
The general problem can be represented as
Maximize z = r
1
m
1
2
m
2
n
m
n
subject to
w
1
m
1
2
m
2
n
m
n
… W
m
1
, m
2
, c, m
n
nonnegative integers
The three elements of the model are
- Stage i is represented by item i, i = 1, 2, c, n.
- The alternatives at stage i are the number of units of item i, m
i
= 0, 1, c,
W
w
i
,
where
W
w
i
is the largest integer less than or equal to
W
w
i
. This definition allows the
solution to allocate none, some, or all of the resource W to any of the m items. The
return for m
i
is r
i
m
i
.
3
The knapsack problem is also known in the literature as the fly-away kit problem (determination of the
most valuable items a jet pilot takes on board) and the cargo-loading problem (determination of the most
valuable items to be loaded on a navy ship). It appears that the three names were coined to ensure equal
representation of three branches of the armed forces: army, air force, and navy!
476 Chapter 12 Deterministic Dynamic Programming
- The state at stage i is represented by x
i,
the total weight assigned to stages (items)
i, i + 1, c, and n. This definition recognizes that the weight limit is the only
constraint that binds all n stages.
4
Define
f
i
1 x
i
2 = maximum return for stages i, i + 1, and n, given state x
i
The most convenient way to construct the recursive equation is a two-step procedure:
Step 1. Express f
i
(x
i
) as a function of f
i
1 x
i + 1
2 as follows:
f
n + 1
1 x
n + 1
2 K 0
f
i
1 x
i
2 =
min
m
i
= 0, 1, c c
W
w i
d
5 r
i
m
i
i + 1
1 x
i + 1
26 , i = 1, 2, c, n
x
i
… W
Step 2. Express x
i + 1
as a function of x
i
to ensure consistency with the left-hand side of the
recursive equation. By definition, x
i
i + 1
= w
i
m
i
represents the weight used
at stage i. Thus, x
i + 1
= x
i
i
m
i
, and the proper recursive equation is given as
f
i
1 x
i
2 =
max
m
i
= 0, 1, c c
W
w
i
d
5 r
i
m
i
i + 1
1 x
i
i
m
i
2 6, i = 1, 2, c, n
x
i
… W
Example 12.3-
A 4-ton vessel can be loaded with one or more of three items. The following table gives the unit
weight, w
i
, in tons and the unit revenue in thousands of dollars, r
i
, for item i. The goal is to deter-
mine the number of units of each item that will maximize the total return.
Item i w i
r
i
Because the unit weight w
i
and the maximum weight W are integers, the state x
i
assumes
integer values only.
Stage 3. The exact weight to be allocated to stage 3 (item 3) is not known in advance but can
assume one of the values 0, 1,... , and 4 (because W = 4 tons and w
3
= 1 ton). A value of m
3
is feasible only if w
3
m
3
… x
3
. Thus, all the infeasible values (with w
3
m
3
7 x
3
2 are excluded. The
revenue for item 3 is 14m
3
. Thus, the recursive equation for stage 3 is
f
3
1 x
3
2 = min
m
3
= 0, 1, c, 4
514 m
3
4
The definition of the state can be multidimensional. For example, the volume of the knapsack may pose
another restriction. In general, a multidimensional state implies more complex stage calculations. See
Section 12.4.
478 Chapter 12 Deterministic Dynamic Programming
Excel Moment
The nature of DP computations makes it impossible to develop a general computer code that can
handle all DP problems. Perhaps this explains the persistent absence of commercial DP software.
In this section, we present an Excel-based algorithm for handling a subclass of DP problems:
the single-constraint knapsack problem (file excelKnapsack.xls). The algorithm is not data-specific
and can handle problems in which an alternative can assume values in the range of 0 to 10.
Figure 12.3 shows the starting screen of the knapsack (backward) DP model. The screen is
divided into two sections: The right section (columns Q:V) summarizes the output solution. In
the left section (columns A:P), the input data for the current stage appear in rows 3, 4, and 6.
Stage computations start at row 7. (Columns H:N are hidden to conserve space.) The input data
symbols are self-explanatory. To fit the spreadsheet conveniently on one screen, the maximum
feasible value for alternative m
i
at stage i is 10 (cells D6:N6).
Figure 12.4 shows the stage computations generated by the algorithm for Example 12.3-1.
The computations are carried out one stage at a time, and the user provides the basic data that
drive each stage.
Starting with stage 3, and using the notation and data in Example 12.3-1, the input cells are
updated as the following list shows:
Cell(s) Data
D3 Number of stages, N = 3
G3 Resource limit, W = 4
C4 Current stage = 3
E4 w
3
G
r 3
D6:H
m
3
Note that the feasible values of m
3
are 0, 1,... , and 4 =
W
w
3
4
1
, as in Example 12.3-1.
The spreadsheet automatically checks the validity of the values the user enters and issues self-
explanatory messages in row 5: “yes,” “no,” and “delete.”
As stage 3 data are entered and verified, the spreadsheet “comes alive” and generates
all the necessary computations of the stage (columns B through P) automatically. The value
- 1111111 is used to indicate that the corresponding entry is not feasible. The optimum solu-
tion (f
3
, m
3
) for the stage is given in columns O and P. Column A provides the values of f
4,
which equal 0 for all x
3
because the computations start at stage 3 (you can leave A9:A
blank or enter zeros).
FIGURE 12.
Excel starting screen of the general DP knapsack model (file excelKnapsack.xls)
12.3 Selected DP Applications 479
Now that stage 3 calculations are complete, take the following steps to create a perma-
nent record of the optimal solution of the current stage and to prepare the spreadsheet for
next stage:
Step 1. Copy the x
3
-values, C9:C13, and paste them in Q5:Q9 in the optimum solution
summary section. Next, copy the (f
3
, m
3
)-values, O9:P13, and paste them in R5:S9.
Remember that you need to paste values only, which requires selecting Paste Special
from Edit menu and Values from the dialogue box.
Stage 2:
Stage 3:
Stage 1:
FIGURE 12.
Excel DP model for the knapsack problem of Example 12.3-1 (file excelKnapsack.xls)
12.3 Selected DP Applications 481
The cost functions C
1
and C
2
are in hundreds of dollars.
Stage 5. 1 b
5
C
1
1 x
5
- 62 + C
2
1 x
5
4
Optimum solution
x
4
x
5
= 6 f
5
(x
4
) x
5
Stage 4. 1 b
4
C
1
1 x
4
- 42 + C
2
1 x
4
3
2 + f
5
1 x
4
Optimum solution
x
3
x
4
= 4 x
4
= 5 x
4
= 6 f
4
(x
3
) x
4
Stage 3. 1 b
3
C
1
1 x
3
- 82 + C
2
1 x
3
2
2 + f
4
1 x
3
2 Optimum solution
x
2
x
3
= 8 f
3
(x
2
) x
6
Stage 2. 1 b
2
C
1
1 x
2
- 72 + C
2
1 x
3
2
2 + f
3
1 x
2
2 Optimum solution
x
1
x
2
= 7 x
2
= 8 f
2
(x
1
) x
2
Stage 1. 1 b
1
C
1
1 x
1
- 52 + C
2
1 x
1
0
2 + f
2
1 x
1
Optimum solution
x 0
x
1
= 5 x
1
= 6 x
1
= 7 x
1
= 8 f
1
(x
0
) x
1
482 Chapter 12 Deterministic Dynamic Programming
The optimum solution is determined as
x
0
S
x
1
S
x
2
S
x
3
S
x
4
S
x
5
The solution can be translated to the following plan:
Week i
Minimum labor
force (b
i
Actual labor
force (x
i
) Decision Cost
1 5 5 Hire 5 workers 4 + 2 * 5 = 14
2 7 8 Hire 3 workers 4 + 2 * 3 + 1 * 3 = 13
3 8 8 No change 0
4 4 6 Fire 2 workers 3 * 2 = 6
5 6 6 No change 0
The total cost is f
1
12.3.3 Equipment Replacement Model
Machines that stay longer in service incur higher maintenance cost and may be re-
placed after a number of years in operation. The situation deals with determining the
most economical age of a machine.
Suppose that the machine replacement problem spans n years. At the start of
each year, a machine is either kept in service an extra year or replaced with a new one.
Let r(t), c(t), and s(t) represent the yearly revenue, operating cost, and salvage value, re-
spectively, of a t-year-old machine. The cost of acquiring a new machine in any year is I.
The elements of the DP model are as follows:
- Stage i is represented by year i, i = 1, 2, c, n.
- The alternatives at stage (year) i are keep (K) or replace (R) the machine at the
start of year i.
- The state at stage i is the age of the machine at the start of year i.
Given that the machine is t years old at the start of year i, define
f
i
1 t 2 = maximum net income for years i, i + 1, c, and n
The recursive equation is
f
n
1 t 2 = max e
r 1 t 2 - c 1 t 2 + s 1 t + 12 , if KEEP
r 102 + s 1 t 2 + s 112 - I - c 102 , if REPLACE
f
i
1 t 2 = max e
r 1 t 2 - c 1 t 2 + f
i + 1
1 t + 12 , if KEEP
r 102 + s 1 t 2 - I - c 102 + f
i + 1
112 , if REPLACE
f , i = 1, 2, c, n - 1
Example 12.3-
A company needs to determine the optimal replacement policy for a current 3-year-old ma-
chine over the next 4 years 1 n = 42. A 6-year-old machine must be replaced. The cost of a new
machine is $100,000. The following table gives the data of the problem:
484 Chapter 12 Deterministic Dynamic Programming
The solution of the network in Figure 12.5 is equivalent to finding the longest route
(i.e., maximum revenue) from the start of year 1 to the end of year 4. We will use the tabular
form to solve the problem. All values are in thousands of dollars. Note that if a machine is re-
placed in year 4 (i.e., end of the planning horizon), its revenue will include the salvage value, s(t),
of the replaced machine and the salvage value, s(1), of the replacement machine. Also, if in year 4
a machine of age t is kept, its salvage value will be s 1 t + 12.
Stage 4.
K R Optimum solution
t r 1 t 2 + s 1 t + 12 - c 1 t 2 r 102 + s 1 t 2 + s 112 - c 102 - I f
4
(t) Decision
1 19.0 + 60 - .6 = 78.4 20 + 80 + 80 - .2 - 100 = 79.8 79.8 R
2 18.5 + 50 - 1.2 = 67.3 20 + 60 + 80 - .2 - 100 = 59.8 67.3 K
3 17.2 + 30 - 1.5 = 45.7 20 + 50 + 80 - .2 - 100 = 49.8 49.8 R
(Must replace)
4.8 R
Stage 3.
K R Optimum solution
t r 1 t 2 - c 1 t 2 + f
4
1 t + 12 r 102 + s 1 t 2 - c 102 - I + f
4
112 f 3
(t) Decision
1 19.0 - .6 + 67.3 = 85.7 20 + 80 - .2 - 100 + 79.8 = 79.6 85.7 K
2 18.5 - 1.2 + 49.8 = 67.1 20 + 60 - .2 - 100 + 79.8 = 59.6 67.1 K
5 14.0 - 1.8 + 4.8 = 17.0 20 + 10 - .2 - 100 + 79.8 = 9.6 17.0 R
Stage 2.
K R Optimum solution
t r 1 t 2 - c 1 t 2 + f
3
1 t + 12 R 102 + s 1 t 2 - c 102 - I + f
3
112 f 2
(t) Decision
1 19.0 - .6 + 67.1 = 85.5 20 + 80 - .2 - 100 + 85.7 = 85.5 85.5 K or R
4 15.5 - 1.7 + 17.0 = 30.8 20 + 30 - .2 - 100 + 85.7 = 35.5 35.5 R
Stage 1.
K R Optimum solution
t r 1 t 2 - c 1 t 2 + f
2
1 t + 12 R 102 + s 1 t 2 - c 102 - I + f
2
112 f
1
(t) Decision
3 17.2 - 1.5 + 35.5 = 51.2 20 + 50 - .2 - 100 + 85.5 = 55.3 55.3 R
Figure 12.6 summarizes the optimal solution. At the start of year 1, given t = 3 , the optimal
decision is to replace the machine. Thus, the new machine will be 1 year old at the start of year 2,
and
t = 1 at the start of year 2 calls for either keeping or replacing the machine. If it is replaced,
the new machine will be 1 year old at the start of year 3; otherwise, the kept machine will be
2 years old. The process is continued in this manner until year 4 is reached.
12.3 Selected DP Applications 485
The alternative optimal policies starting in year 1 are (R, K, K, R) and (R, R, K, K). The total
cost is $55,300.
12.3.4 Investment Model
Suppose that you want to invest the amounts P
1
, P
2
,... , P
n
at the start of each of the
next n years. You have two investment opportunities in two banks: First Bank pays an
interest rate r
1
, and Second Bank pays r
2
, both compounded annually. To encourage
deposits, both banks pay bonuses on new investments in the form of a percentage of
the amount invested. The respective bonus percentages for First Bank and Second
Bank are q
i 1
and q
i 2
for year i. Bonuses are paid at the end of the year in which the
investment is made and may be reinvested in either bank in the immediately suc-
ceeding year. This means that only bonuses and fresh new money may be invested in
either bank. However, once an investment is deposited, it must remain in the bank
until the end of year n.
The elements of the DP model are as follows:
- Stage i is represented by year i, i = 1, 2, c, n.
- The alternatives at stage i are I
i
and I
i
, the amounts invested in First Bank and
Second Bank, respectively.
- The state, x
i
, at stage i is the amount of capital available for investment at the start
of year i.
We note that I
i
= x
i
i
by definition. Thus
x
1
= P
1
x
i
= P
i
i - 1, 1
I
i - 1
i - 1, 2
1 x
i - 1
i - 1
2
= P
i
i - 1, 1
i - 1, 2
2 I
i - 1
i - 1, 2
x
i - 1
, i = 2, 3, c, n
The reinvestment amount x
i
includes only new money plus any bonus from invest-
ments made in year i - 1.
Define
f
i
1 x
i
2 = optimal value of the investments for years i, i + 1, c, and n, given x
i
K (t 5 2) K (t 5 3)
(t 5 3)
(t 5 1) R
R
R (t 5 1) K (t 5 2) K
Sell
Year 1 Year 2 Year 3 Year 4
FIGURE 12.
Solution of Example 12.3-
12.3 Selected DP Applications 487
Optimum solution
State f
4
(x
4
) I
4
x
4
1.108x
4
Stage 3.
f
3
1 x
3
2 = max
0 … l
3
… x
3
5 s
3
+ f
4
1 x
4
where
s
3
2
2
2 I
3
2
x
3
= .00432I
3
+ 1.1621x
3
x
4
= 2000 - .005I
3
+ .026x
3
Thus,
f
3
1 x
3
2 = max
0 … I 3
… x 3
5 .00432I
3
+ 1.1621x
3
+ 1.108 12000 - .005I
3
+ 0.026x
3
= max
0 … I
3
… x
3
52216 - .00122I
3
+ 1.1909x
3
Optimum solution
State f
3
(x
3
) I
3
x
3
2216 + 1.1909x
3
Stage 2.
f
2
1 x
2
2 = max
0 … I
2
… x
2
5 s
2
+ f
3
1 x
3
where
s
2
3
3
2 I
2
3
x
2
= .006985I
2
+ 1.25273x
2
x
3
= 2000 - .005I
2
+ .022x
2
Thus,
f
2
1 x
2
2 = max
0 … I
2
… x
2
5 .006985I
2
+ 1.25273x
2
+ 2216 + 1.1909 12000 - .005I
2
+ .022x
2
= max
0 … I 2
… x 2
5 4597.8 + .0010305I
2
+ 1.27893x
2
Optimum solution
State f
2
(x
2
) I
2
x
2
4597.8 + 1.27996x
2
x
2
Stage 1.
f
1
1 x
1
2 = max
0 … I 1
… x 1
5 s
1
+ f
2
1 x
2
488 Chapter 12 Deterministic Dynamic Programming
where
s
1
4
4
2 I
1
4
x
1
= .01005I
2
+ 1.3504x
1
x
2
= 2000 - .005I
1
+ .023x
1
Thus,
f
1
1 x
1
2 = max
0 … I
1
… x
1
5 .01005I
1
+ 1.3504x
1
+ 4597.8 + 1.27996 12000 - .005I
1
+ .023x
1
= max
0 … I 1
… x 1
5 7157.7 + .00365I
1
+ 1.37984x
1
Optimum solution
State f
1
(x
1
I
1
x
1
7157.7 + 1.38349x
1
Working backward and noting that I
1
= 4000, I
2
= x
2
, I
3
= I
4
= 0, we get
x
1
x
2
x
3
x
4
The optimum solution is thus summarized as
Year Optimum solution Decision Accumulation
I
1
= x
1
Invest x 1
= $4000 in First Bank s 1
I
2
= x
2
Invest x
2
= $2072 in First Bank s
2
I
3
Invest x
3
= $2035.22 in Second Bank s
3
I
4
= 0
Invest x
4
= $2052.92 in Second Bank s
4
Total accumulation = f 1
1 x 1
2 = 7157.7 + 1.38349 140002 = $12,691.66 1 = s 1
12.3.5 iPvGPVQr[MQdGls
DP has important applications in the area of inventory control. Chapters 13 and 16
present some of these applications. The models in Chapter 13 are deterministic, and
those in Chapter 16 are probabilistic. Other probabilistic DP applications are given in
Chapter 24 on the website.
- 4 PRoBLeMoFDiMenSionALity
In all the DP models presented in this chapter, the state at any stage is represented by
a single element. For example, in the knapsack model (Section 12.3.1), the only restric-
tion is the weight of the item. More realistically in this case, the volume of the knapsack
