Chapter 4 - Simplex method Flashcards
When, and by who, was the simplex method developed?
1947 by George Dantzig
The simplex method is widely used. Why?
Simplex is a solver for LP problems. LP problems occur in a large number of applications. Therefore, LP is common.
Simplex is used specifically, because it is very efficient. GIven the standard form model, simplex will efficiently determine how to progress the search in a manner that only explore a very small subset of the total amount of possible solutions, namely the corner points feasible solutions only.
“what” is the simplex method?
An algebraic procedure, but with underlying geometric concepts.
What is the “optimality test”?
Optimality test goes like this:
Consider any LP problem that posesses at least one optimal solution. If a CPF solution has no adjacent CPF solutions that are BETTER, then this current CPF solution is optimal.
ex:
Z = 30 + 3x_1 - 5/2 x_4
We see here that increasing x_1 vs x_4 will yield the answer. Since x_4 has a negative rate of change/improvement, the task becomes trivial. However, formally, x_1 is the “entering basic variable”
Geometrically, we say that Z(a, b) is not optimal because moving in the direction of x_1 will increase Z and do nothing with x_4. Extra geometrically we say “moving along the edge between the current point and some other point”.
In conclusion, the optimality test is about: If there are no adjacent BFS that are better than the current BFS, then the current BFS is optimal.
There are 6 solution concepts, name them
1) The simplex method focus solely on CPF solutions. For any problem that contains an optimal solution, we only have to find the “best” CPFS. This is much of the power of simplex. We only consider an extremely small subset of all possible points.
2) Iterative process
3) Whenever possible, initialization should choose the origin as the starting point, meaning all decision variables equal to zero.
4) Given a CPFS it is MUCH QUICKER to find information about its adjacent CPFS than any other CPFS. Therefore, we always move to adjacent CPFS. We dont “search” for the points, we calculate them using the shared constraint boundaries.
5) When a CPFS is found, simplex considers the rate of change of any possible adjacent direction. Then it picks the best one.
6) The optimality test at some CPFS consists of checking whether there are ANY possible adjacent edge/direction that gives Z a positive rate of change.
Given an LP problem in “our standard form”, what is the first step?
The first step is to convert the model into a set of linear equations by using slack variables. This is simple:
Say we have: x_1 <= 10
Then we must add a variable, x_2, such that the shite becomes an equality:
x_1 + x_2 = 10
We must also include the non-negativity constraint.
When we have included slack variables, what do we call our model+
augmented (utvided).
if a slack variable equals 0 in the current solutions, what does this mean?
It means that the solution lies directly on the constraint boundary for the corresponding functional constraint.
if a slack variable is greater than 0, what does it mean+
It means that the solution is within the feasible region.
What is an augmented solution?
An augmented solution is any solution that is augmented to include the slack variables as well.
What is a basic solution?
A basic solution is a corner-point solution that is augmented to include the slack variables
What is a BFS?
BFS = Basic Feasible Solution. Augmented corner-point feasible solution. CFS with slack variables.
Elaborate on degrees of freedom
Degrees of freedom refer to the number of variables that exceeds the number of equations in our model.
They are called degrees of freedom since they are essentially free to be any value and the system of equations will be solved. However, we are not interested in a solution to the system. We want the best solution.
Although we can select any value for the free variables, we force them to be zero. This is because we want to consider corner-points.
The number of free variables are related to the number of non-basic variables, in that they are the same number. What variables we choose to be the free variables is completely arbitrary. However, it is worth noting that some variables as 0 will produce systems that cannot be solved. In such cases, we must try something else (try another combination of free variables).
The variables that are chosen to be “free variables” are called non-basic variables. Simplex set them to be 0.
What are the properties of basic solutions?
1) Each variable in a basic solution will be either basic or non-basic.
2) The number of basic variables equals the number of functional constraints (now equations).
3) The nonbasic variables are set equal to zero.
4) The values of the basic variables are obtained as the simultaneous solution of the system of equations.
5) If the basic variables satisfy non-negativity, the basic solution is BFS (Basic Feasible Solution)
How to tell if two BFS are adjacent?
Two basic-feasible solutions are adjacent if their set of non-basic variables is equal for all but one variable.
This is due to the fact that if we want to move to an adjacent BFS, then one, and only one, non-basic variable will change. If not, they are not adjacent.
What happens when we move from one BFS to another?
One of the non-basic variables will be turned into a basic variable, and one of the basic variables will be turned into a non-basic variable.
Give the outline of the simplex method
1) Initialization: Identify an initial solution
2) Perform the optimality test to determine if this init solution is optimal. If YES, then STOP. if NO then CONTINUE
3) Step 1 of iteration) Determine which direction to move.
4) Step 2 of iteration) Determine where to stop to reach the next solution.
5) Step 3 of iteration) Solve system for this new equation.
6) Return to the optimality test
WHat is the difference between original form and augmented form of a model?
THe augmented form includes slack variables. This means that the augmented matrix contains equalities and not inequalities etc.
Elaborate on the initialization step of the algebraic simplex method
Recall this:
A problem/model consisting of n variables and r constraints that is in our standard form, will introduce slack variables equal to the number of constraints. Therefore, the augmented model will require (n+r) variables, but the number of constraints (now turned to equations) remains the same.
We know that the number of non-basic variables is then (n+r)-r=n. This means, we will always have a set of nonbasic variables that has the same size as the number of decision variables from the original problem. This allows us to always pick exactly the set of decision variables as our initial starting point, given that there are no constraints violating this.
When we pick this init point as the origin (since we set value equal to 0), we then solve the remaining system to get the values of the slack variables.
On the degrees of freedom: Recall that we get n degrees of freedom because there are n more variables than equations. Therefore, we can solve the system by choosing ANY value for these free variables.
when we solve the augmented model using Gauss, what are we actually doing/finding?
We are finding out how much gap there is between the point given by the values of the decision variables and every constraint-border, given by the slack variables.
At first, we want to pick (0,0,…) as the nonbasic variable set. Since there are n decision variables, we get n nonbasic variables. Therefore, we are finding out how much slack there is in each constraint from this point.
Elaborate on the optimality test
Say we have been given the objective function:
Z = 3x_1 + 5x_2
If we are at point (0,0) it is easy to see that Z is 0.
Since both variables give a positive contribution by increasing them, we see that Z is currently not optimal.
Elaborate on determining the direciton of movement (Step 1 of iteration)
We consider the current shape/form of the objective function. The objective function will change along with the procedure.
We look at the different parts of the objective function, and find the variable that has the largest positive coefficient. This represent the direction with the greatest rate of improvement.
The important part is that we always only change one non-basic variable by moving to an adjacent node. This means that we control the movement. If we have 2 decision variables that are non-basic, moving in either of those directions will only change one of them. Likewise, if we have a combination of slack variables and decision variables as non-basic variables, if we move to an adjacent node, we only change on non-basic variable. This is key. We choose one non-basic to be “entering basic”, and force the ohthers to remain at zero.
IMP: We choose one NONBASIC variable to increase/move along. The variable we choose is called the “entering basic variable”, because when we have moved as much as we can in this direction, this variable has turned to a basic variable.
Elaborate on how we determine how “far” to move in a speicfic direction given that we have chosen a variable to move along
Recall that we have chosen a nonbasic variable to move along. We want to move all the way until we reach a constraint boundary. The idea is to move until one of the basic variables becomes 0. At this point, the decision variables will indicate a point that is a BFS (CPF + augmented), and it will be better than the point we previously explored, according to the objective function.
Given the nonbasic variable we have chosen, we want to check how far it can be changed. Therefore, we find all constraints involving the chosen nonbasic variable, and see how far it can be increased without violating the constraint. to do this, we use the non-negativity constraints.
Say the equation is like this, and we have chosen x_2 as the nonbasic to increase:
2x_2 + x_4 = 12
Then we use the nonnegativty constraint of x_4 >= 0
Thus we get:
x_4 = 12 - 2x_2 >= 0
This gives us…:
12 >= 2x_2
6 >= x_2
Therefore, we know that x_2 must be smaller than, or equal to, 6.
We do this for every constraint involving x_2.
Then we pick the smallest number, and use it as the max length we can move/increase the variable.
We also see how choosing x_2=6 makes x_4 = 0. Therefore, we want to pick x_4 as the leaving basic variable.
Recall that x_2 is now no longer a nonbasic variable.
IMPORTANT: When we check the constraints to figure out how far we can move, we must always use the most up-to-date equations given by gauss. Or do we?
Consider the second step of each iteration in the simplex method. Recall what it is.
Elaborate on how to perform it
The second step of each iteration in the simplex method is about figuring out the value of the entering basic variable.
We perform this step by considering every constraint that includes the “entering basic variable” and utilizing the fact that all variables should be non-negative.
IMP: Since we have solved the system prior, we have a linear system of equations where every basic variable is a function of the non-basic variables. This makes us able to use the fact that this variable should be nonneg. We then take every equation where the “entering basic variable” is included, and applies the fact that the basic variable in this equation should be non negative. This allows us to find the largest allowed value for the entering basic variable from this specific constraint.