Lecture 1

Question 1

Definition:
Numerical (Mathematical) Optimization

Answer 1

Finding the best possible solution using a mathematical problem formulation and a rigorous / heuristic numerical solution method

Question 2

The general formulation of an optimization problem
consists of

Answer 2

*The variables (also called decision variables, degrees of freedom, parameters, …)
*
An objective function
*
A mathematical model for the description of the system to be optimized
*
Additional restrictions on the optimal solution, including bounds of the variables.

( mathematical model + additional restrictions = constrains )

Question 3

objective function

Answer 3

The objective function describes an economical measure (operating costs, investment costs, profit, etc.), or
technological, or …

Question 4

additional constraints

Answer 4

The additional constraints (mostly linear inequalities) result, for instance, from:
- plant or equipment specific limitations (capacity, pressure, etc.)
- material limitations (explosion limit, boiling point, corrosivity, etc.)
- product requirements (quality, etc.)
- resources (availability, quality, etc.)

Question 5

Classification of Optimization Problems

Answer 5

• Linearity of objective function and constraints:
• Linear (LP) versus nonlinear programs NLP)
• NLPs can be convex or nonconvex, smooth or nonsmooth
• Discrete and/or continuous variables:
• Integer programs (IP) and mixed integer programs (MIP or MILP and MINLP, respectively)
• Time dependence:
• Dynamic optimization or optimal control programs (DO or OCP)
• Stochastic or deterministic models and variables:
• Stochastic programs, semi infinite optimization, …
• Single objective vs multi objective , single level vs multi level

Question 6

optimization problem

Answer 6

An optimization problem: mathematical formulation to find the best possible solution out of all feasible
solutions. Typically comprising one or multiple objective function(s), decision variables, equality constraints
and/or inequality constraints.

Question 7

algorithm

Answer 7

An algorithm is a procedure for solving a problem based on conducting a sequence of specified actions.
The terms ‘ algorithm ’ and solution method ’ are commonly used

Question 8

solver

Answer 8

A solver is the implementation of an algorithm in a computer using a programming language. Often, the
terms ‘ solver ’ and software ’ are used

Question 9

Some Issues with Optimization

Answer 9

• Not a button press technology
• Need expertise for model formulation, algorithm selection and tuning, checking results, …
• “Optimizer’s curse”: solution using good algorithm and bad model will look better than what it is
• Random error : if the model has a random error and we optimize, the true objective value of the solution found will be
  worse than the calculated one
• If model allows for nonphysical solution with good objective value, good optimizer will pick such
  On the other hand, model has to just lead in correct direction, not be correct
• Many engineering (design) problems are nonconvex , but global algorithms are inherently very expensive
• Often optimal solution at constraint, thus tradeoff good vs. robust solution

Question 10

Nonlinear Optimization Problem (Nonlinear Program, NLP)
General formulation:

Answer 10

min𝒙∈𝐷 𝑓(𝒙)

𝑐𝑖(𝒙) = 0 𝑖 ∈ 𝐸
𝑐𝑖(𝒙) ≤ 0 𝑖 ∈ 𝐼

Question 11

Formulation and Solution of Optimization Problems

Answer 11

1. Determine variables and phenomena of interest through systems analysis
2. Define optimality criteria: objective function(s) and (additional) constraints
3. Formulate a mathematical model of the system and determination of
   degrees of freedom (number and
4. Identify of the problem class (LP, QP, NLP, MINLP, OCP
5. Select (or develop) a suitable algorithm
6. Solve the problem using a numerical solve r
7. Verify the solution through sensitivity analysis, understand results, …