2.3. Optimization Methods

Question 1

What are optimization models in machine learning?

Answer 1

Mathematical formulations that aim to find the best solution from a set of feasible solutions

Question 2

What are constraints in optimization models?

Answer 2

Conditions that the solution must satisfy, limiting the feasible region.

Question 2

What is linear programming?

Answer 2

An optimization technique where the objective function and constraints are linear.

Question 3

What is the objective function in optimization?

Answer 3

A function that needs to be maximized or minimized based on the problem requirements.

Question 4

What is the Simplex method?

Answer 4

A popular algorithm for solving linear programming problems.

Question 5

What is non-linear programming?

Answer 5

Optimization where the objective function or constraints are non-linear.

Question 6

What is convex optimization?

Answer 6

A subset of optimization where the objective function is convex, ensuring a global minimum.

Question 7

What is gradient descent?

Answer 7

An iterative optimization algorithm used to minimize a function by moving in the direction of the steepest descent.

Question 8

What is the role of Lagrange multipliers in optimization?

Answer 8

They are used to find the local maxima and minima of a function subject to equality constraints.

Question 9

What is the difference between global and local optimization?

Answer 9

Global optimization seeks the best solution across the entire feasible region, while local optimization finds the best solution within a limited neighborhood.

Question 10

What is integer programming?

Answer 10

An optimization technique where some or all decision variables are constrained to be integers.

Question 11

What is the primary goal of optimization methods in machine learning?

Answer 11

To optimize a given cost function during the learning stage.

Question 12

Name one type of optimization method discussed in the document.

Answer 12

Support Vector Machines (SVM).

Question 12

What is a key characteristic of Artificial Neural Networks (ANN) in the context of optimization?

Answer 12

They are biologically inspired and involve optimizing weights and biases.

Question 12

What distinguishes optimization methods from other predictive modeling methods?

Answer 12

Optimization methods aim to optimize the entire learning function, not just a part of it.

Question 13

What are linear methods in optimization?

Answer 13

Techniques that involve linear least squares and regression analysis.

Question 13

What is the purpose of a cost function in optimization?

Answer 13

To measure how well a model performs, guiding the optimization process.

Question 14

What is the role of residuals in linear methods?

Answer 14

Residuals represent the difference between observed and predicted values, used to assess model accuracy.

Question 15

How do Support Vector Machines (SVM) optimize their function?

Answer 15

By finding the hyperplane that maximizes the margin between different classes.

Question 16

What is a common application of optimization methods in finance?

Answer 16

Portfolio optimization to maximize returns while minimizing risk.

Question 17

What is the significance of the learning function in optimization?

Answer 17

It represents the relationship between input features and the target variable, which needs to be optimized.

Question 18

What is the difference between supervised and unsupervised optimization methods?

Answer 18

Supervised methods use labeled data, while unsupervised methods work with unlabeled data.

Question 19

What is a common challenge in optimization?

Answer 19

Avoiding local minima to ensure the global optimum is found.

Question 20

Why is it important to evaluate the performance of optimization methods?

Answer 20

To ensure that the model generalizes well to unseen data and meets the desired objectives.

Question 20

What is the role of hyperparameters in optimization methods?

Answer 20

Hyperparameters are settings that govern the optimization process and can significantly affect model performance.

Question 21

What is the relationship between optimization and machine learning?

Answer 21

Optimization is a fundamental aspect of training machine learning models, as it directly impacts their accuracy and efficiency.