Download Linear Programming: Minimization with Constraints and more Study notes Physics in PDF only on Docsity!
7. Constrained Optimization
7.1 Fundamentals, Definitions and Terminology
The discussion so far has focussed on unconstrained optimization, that is with no restrictive limits being placed on the values to which the variables may be set. In practice, there are often physical constraints which prevent complete optimization of the problem, e.g. dimension lim- its in certain areas. In mathematical terms the problem becomes:
minimize , subject to , (7.1) , ,
where for independent constraints.
Constraints are said to be active when they impose a direct restriction on small variations of x. Thus, equality constraints are always active. If , an inequality constraint is active. If , an inequality constraint is inactive. An inequality constraint is violated , if .
Figure 7.1 illustrates different cases: in (a), none of the constraints are active near the mini- mum; in (b) and (d), only one of three constraints is active at the minimum; and in (c), two constraints are active at the minimum.
As Figure 7.1 demonstrates, identifying the feasible region graphically and superimposing objective function contours on this diagram can help to identify which constraints will be active at the optimum, and, in some cases, it may even be possible to identify where the opti- mum is by inspection!
Note that if the constraints are independent, it is impossible to have more that n constraints active at once. Also, if it was possible to know the active and inactive constraints at the opti- mum in advance, then we could ignore the inactive constraints and treat the active ones as equality constraints.
By suitable rearrangement and substitution of variables it is often possible to eliminate equal- ity constraints altogether (by eliminating a variable) and to rearrange inequality constraints into simple limits on a variable. This greatly reduces the chances of a search algorithm getting stuck, as exploratory searches along other variables are then in directions which can’t activate constraints.
f ( x ) x = ( x 1 , x 2 ,... , xn ) T hi ( x ) = 0 i =1 2,..., , m ′ gi ( x ) ≤ 0 i = m ′ +1,... , m m ≤ n
gi ( x ) = 0 gi ( x ) < 0 gi ( x ) > 0
Figure 7.1 : Examples of Minima with Constraints.
x 1
x 2
x 1
x 2
x 1
x 2
x 1
x 2
(a) (b)
(c) (d)
7.2 Linear Programming
An important class of constrained optimization problems are those for which the problem functions (objective and constraints) are linear functions of the control variables:
minimize , subject to , (7.2) and ,.
A number of methods have been developed for tackling such problems, of which one of the best known is the Simplex Method.
f ( x ) = c T^ x x = ( x 1 , x 2 ,... , xn ) T a iT x − bi = 0 i =1 2,..., , m xj ≥ 0 j =1 2,..., , n
7.2.2 Example: Setting up a Linear Program
The following is an example of a linear problem that we want to reduce to the form described above:
An oil refinery has three sources of crude oil:
- a heavy crude that costs $10/barrel;
- a medium crude that costs $17/barrel;
- a light crude that costs $20/barrel.
The refinery produces petrol and heating oil from crude in the following amounts per barrel:
The refinery is contracted to supply 200000 barrels of petrol and 100000 barrels of heating oil. How much of each type of crude should the refinery buy to minimize its costs?
Give the following variable names to the amount of each crude bought:
Heavy crude: ; medium crude: ; light crude:.
The contract to supply then becomes two constraints:
While the objective function (cost) to be minimized is:
Thus, the problem can be written as:
minimize , subject to , (7.3) and ,
where
; ;.
Petrol Heating Oil Heavy Crude 0.3 0. Medium Crude 0.5 0. Light Crude 0.7 0.
x 1 x 2 x 3
0.3 x 1 + 0.5 x 2 + 0.7 x 3 = 2 × 10 5 0.7 x 1 + 0.5 x 2 + 0.3 x 3 = 1 × 10 5
f ( x ) = 10 x 1 + 17 x 2 + 20 x 3
f ( x ) = c T^ x A x = b xj ≥ 0
A 0.3^ 0.5^ 0.
= b^2
×^5
1 × 10 5
= c T = 10 17 20
We know that the optimum must lie at one of the extreme points. For this simple problem we can find these points by hand:
7.3 The Simplex Method
The feasible set in the example in section 7.2.1 was simply a line, but in higher dimension problems the feasible set is a complicated polyhedron with edges, faces and vertices. The Sim- plex Method exploits the fact that the optimum of a linear program is at one of the extreme points of the feasible set, i.e. one of its vertices:
- The Simplex Method first finds one of the vertices, and then moves from vertex to vertex along the edges of the feasible set.
- For each move the Simplex Method moves along an edge that reduces the value of the objective function.
- Eventually it finds one of the extreme points where the objective function increases along every edge leading away from it — this is the optimum.
Finding the first vertex is called Phase 1 of the Simplex Method, which will be explained later. First, we will look at the remainder of the algorithm, known as Phase 2.
reach another vertex, when either or become zero. Looking again at the equations above:
So, as increases, only decreases (moves towards zero). reaches 0 when , at which point.
Thus, the new vertex and the objective function , reduced from the original 14.
There is now a new set of basic variables, and. As became a basic variable it is called the entering variable , and as is no longer a basic variable it is called the leaving variable.
To check whether this new vertex is the optimum, we now repeat the whole process. But there is a problem. The original form of the constraint equations were given by:
Note that, for the first step, the coefficients of the initial basic variables formed an identity matrix:
This was crucially important to being able to calculate the reduced costs, and is called the canonical form of the problem.
Now the basic variables are and , we need to convert the problem to canonical form. We can do this by making the coefficients of these variables the identity matrix by rearranging the initial constraint equations.
If the second equation is replaced by the first plus the second (i.e. we perform Gaussian elimi- nation ), then the constraint equations become:
which is in the form required.
We now want to be able to calculate the reduced costs associated with or increasing from
- If was increased, with , then this would force:
x 2 x 3
x 4 x 2 x 2 x 4 = 4 x 3 = 6 x T^ = ( 0 0 6 4, , , ) c T^ x = 10
x 3 x 4 x 4 x 2
2 x 1 + x 2 + x 4 =^4 x 1 + x (^) 3 − x 4 =^2
x (^) 2 x (^) 3
x 1 x 4
x 3 x 4
2 x 1 + x 2 + x 4 =^4 3 x 1 + x (^) 2 + x (^) 3 =^6
x 1 x 2 x 2 x 1 = 0
The objective function then becomes:
Thus, the coefficient of , the reduced cost. As this is positive, we do not follow this edge.
If was increased, with , then this would force:
The objective function then becomes:
So, the coefficient of , the reduced cost.
Thus, both reduced costs are positive, so moving away from this vertex will increase the objective. Therefore this vertex is the optimum.
7.3.2 The Tableau
The Simplex Method can be written down in a systematic way by generating a matrix called the tableau , which contains all the information about the linear program.
At the start, when and were the basic variables, the tableau formed from A , b and c is as follows:
The current objective function value can be found by eliminating the components of the basic variables, and , from the final row.
Subtracting the first equation and the second equation from the third:
You should recognise all the entries in the final row:
- The values 2 and −1 are the reduced costs for the corresponding variables (as found earlier), and.
- The current objective function value appears in the bottom right-hand corner (multiplied by −1!).
x 2 r 2 =+
x 1 x 2 = 0
x 1 r 1 =+
x 2 x 3
T 0^ A^ b c T^ 0
x 2 x 3 3 × 1 ×
T 1
r 1 = +2 r 4 =− 1
7.3.3 Organisation of a Simplex Step
- Convert the problem to canonical form. In the tableau this is done by elimination from the matrix A so that the coefficients of the basic variables form an identity matrix. It is impor- tant that this step is possible, i.e. that the constraint equations are independent.
- Calculate the reduced costs associated with each of the free variables. In the tableau this is done by eliminating from the objective function any component that corresponds to a basic variable. Steps 1 and 2 are usually done at the same time.
- If , stop. The current solution is the optimum. Otherwise find the most negative component and let the corresponding increase from 0. This is the entering variable.
- Calculate the ratio of the final column of the tableau (the current values of the basic vari- ables) to the i th column of the tableau — this shows how quickly each variable will decrease as increases. The smallest of these ratios (ignoring the negative ones) will iden- tify the first basic variable to reach 0 as increases. This is the leaving variable. (If all the ratios are negative, the problem is unbounded , and the minimum objective function value will be .)
This completes a step of the Simplex Method
7.3.4 Slack Variables
The standard form for linear programming (equation (7.2)) assumes that all the constraints are equality constraints. This is, of course, not necessarily the case. However, it is possible to con- vert linear inequality constraints of the form
(7.4)
into a linear equality constraint by introducing a slack variable defined by
, (7.5)
and as long as the original constraint is satisfied. Thus, by the introduction of another control variable, the inequality constraint can be converted into the standard form:
;. (7.6)
Similarly, an inequality constraint of the form
(7.7)
can be converted into standard form as:
;. (7.8)
ri
ri ≥ 0 ∀ i ri xi
xi xi
− ∞
a iT x ≥ bi si si = a iT^ x − bi si ≥ 0
a iT x − si = bi si ≥ 0
a iT x ≤ bi
a iT x + si = bi si ≥ 0
7.3.5 Phase 1 of the Simplex Method
Let us now return to the question of finding an initial feasible solution — Phase 2 of the Sim- plex Method assumes one has been found. Phase 1 finds an initial feasible solution by creating and solving a new problem.
Consider the earlier example. Phase 1 of the Simplex Method creates new variables, and , and inserts them into so that they are the initial basic variables of the new problem (after ensuring the signs are reversed in any equation that has a negative right hand side):
Phase 1 also creates a new objective function:
; i.e.
This problem is then solved using Simplex Method Phase 2.
The optimum for this new problem will be at = 0, and the remaining vari- ables will now be a feasible solution to the original problem. If the optimum is reached when , then there is no feasible solution to the original problem.
The initial tableau for this Phase 1 problem (which is already in canonical form) is:
Note that and are the basic variables. The reduced costs can be found by elimination:
Obviously must be the entering variable, as its reduced cost is the most negative. The ratios of the current values of and to column 1 are then:
This shows that both and will reach 0 at the same time, when , and so either can be the leaving variable. If we choose , then the new basic variables are and , and we are ready to start a new Simplex step.
m = 2 x 5 x 6 A x = b
2 x 1 + x 2 + x 4 + x 5 =^4 x 1 + x (^) 3 − x 4 + x 6 =^2
f ( x ) = c T x = x 5 + x 6 c T = 0 0 0 0 1 1
f ( x ) = 0 x 5 = x 6
f ( x ) > 0
T 0
x 5 x 6
T 1
x 1 x 5 x 6
4 ⁄ 2 2 ⁄ 1
x 5 x 6 x 1 = 2 x 6 x 1 x 5
