Optimisation and Regression Flashcards
What are independent variables
Independent variables are input variables that can be freely changed or manipulated, e.g. age of a person
What are dependent variables
Dependent variables are output that cannot be changed freely without altering the inputs, e.g. temperature of a room after adjusting the thermostat
What are categorical variables
A categorical variable is a characteristic that in not quantifiable, e.g. type of fruit
What are nominal variables
A nominal variable is a categorical variable where there is no natural order or hierarchy, e.g. gender of person
What are ordinal variables
An ordinal variable is a categorical variable where there is a natural order or hierarchy, e.g. Education level
What are numeric variables
A numeric variable is a characteristic that is quantifiable, e.g. Number of items sold
What are continuous variables
A continuous variable is a numeric variable that takes real values of infinite precision, e.g. Time taken to complete a task (in seconds)
What are discrete variables
A discrete variable is a numeric variable that takes finite options, e.g. number of cars in a car park
What is a function
A function defines the relationship between inputs and output(s)
How is a function like a mathematical model
A function is like a mathematical model because it tells how the output(s) would vary given a change in inputs
What is the definition of optimization
Optimization is the process of finding the best option or solution form a set of alternatives
What is the goal of optimization
The goal of optimization is to find a specific vector of input or independent variables that produce a desired output or dependent variable value
What are the input variables in optimization
The input variables in optimization are the independent variables, which can be free changed or manipulated
What is the dependent variable in optimization
The dependent variable in optimization is the output, which cannot be changed freely without altering the inputs
What is the objective of optimization
The objective of optimization is to maximize or minimize a particular objective function, which is a mathematical expression that represents the relationship between the input and output variables
What are some examples of optimization problems
Finding the most profitable investment portfolio given a set of assets and market conditions
Designing an aeroplane wing that minimizes drag and maximise lift
Determining the optimal production schedule for a manufacturing plant to minimize costs and maximise profits
What are some common techniques used in optimization
Trial and Error
Geometric
Metaheuristics
Data-driven
What is the trial and error approach
The trial and error approach involves trying different solutions and observing the outcome to find the best option
What is the geometric approach
The geometric approach involves generating new points using some form of geometric knowledge, such as rotating or reflecting a point
What is the calculus approach
The calculus approach involves evaluating the function and its derivatives to direct the search in the direction that minimises or maximises the function
What are metaheuristics
Metaheuristics are problem-solving techniques that draw inspiration from natural processes to find solutions. Examples include simulated annealing, genetic algorithms, and particle swarm optimisation
What is the data-driven approach
The data-driven approach involves using information gathered from previous solutions to improve search for the best option. This can include machine learning or statistical modelling to make predictions based on past data
When should you use trial and error
When the problem is simple, and the number of possible solutions is limited. It is also appropriate when the cost of failure is low and the outcome of each trial can be easily observed
When should you use geometric
When the problem involves finding an optimal geometric arrangement or position. It is also appropriate when the optimization problem can be formulated as a geometric optimization problem, such as finding the shortest distance between two points
When should you use calculus
When the optimization problem can be formulated as a mathematical function and the function and its derivatives can be easily evaluated. It is also appropriate when the optimization problem involves finding the maximum or minimum value of a continuous function
When should you use metaheuristics
When the optimization problem is complex and the search space is large and/or multi-dimensional. Metaheuristics can be used when the optimization problem is non-linear or when the function to be optimized is not known
When should you use data-driven
When there is a large amount of data available and patterns can be extracted from the data. It is also appropriate when the optimization problem is complex and the relationship between the input and output variables is not well-understood
Is finding the global optimum always feasible in optimization
No, finding the global optimum can be difficult or even impossible, especially when dealing with complex problems with a large number of variables and constraints
What is a local optimum
A local optimum is the best solution within a certain region of the search space
Is a local optimum always the best
No, a local optimum may not always be the best solution for the overall problem
How can local optima be found
Local optima can be found using various optimization algorithms, such as geometrics or metaheurestics
Why are we usually happy with a solution that is “good enough”
Finding the global is often too difficult, so we settle for a solution that is “good enough” to meet requirements of the problem
What is regression
Regression is the process of predicting a premise value for the output (or dependent) variable based on a value of the input (or independent) variable
What is linear regression
Linear regression is a type of regression where the relationship between the input variable(s) and the output variable is modelled as a straight line
What are the characteristics of a line in linear regression
A line in linear regression is straight with no bends, has no thickness, and extends to positive or negative infinity
What is the mathematical function for a line in linear regression
y = mx + c, where y is the output variable, x is the input variable, m is the slope of the line, and c is the y-intercept of the line
How is linear regression used for prediction
In linear regression, the input variable(s) are used to predict the output variable by finding the best-fitting line that minimizes the distance between predicted values and the actual values. Once the line is established, the value of the output variable can be predicted for any given value of the input variable
How do you generalise a linear equation for 1-D input space
y = f(x,m,c) = mx + c
How do you generalise a linear equation for 2-D input space
y = f(x0, x1, m0, m1, c) = m0x0 + m1x1 + c
How do you generalise a linear equation for N-D input space (Check notes)
y = f(x0, x1, …, xn-1, m0, m1, …, mn-1, c) = m0x0 + m1x1 + … + mn-1xn-1 + c
What does the error or residual represent in a linear regression context
The error on residual represents the difference between the predicted value and the actual value of the dependent variable for a particular data point in linear regression
How do we calculate the error or residual for a single data point
To calculate the error or residual for a single data point, we subtract the predicted value from the actual value of the dependent value
Why do we square the error or residual in linear regression
We square the error or residual in linear regression to avoid cancellation of positive and negative errors and to emphasize larger errors, which helps in identifying and reducing them
What does a negative error or residual value indicate in linear regression
A negative error or residual value indicates that the predicted value is higher than the actual value of the dependent variable for a particular data point in linear regression
What is the error or residual value in the given example (check notes), where the estimated value is y = 1000 and the measured value is y = 1025
The error or residual value is e = 1025 - 1000 = 25, and the squared error or residual is e^2 = 625
What is the model estimate
The model estimate is the predicted value of the output variable based on the input variable(s)
What is the true measurement
The true measurement is the actual value of the output variable
General Error Function (for one point)
Check notes
General Error Function (for k points)
Check notes
General Error Function (for k points)
Check notes
What is a neuron
A neuron is the basic unit of a brain that works as an information messenger
How does a neuron work
A neuron receives electrical impulses and chemical signals through its axon and forwards them - if it is excited enough - to other neurons through its dendrites
What happens when neurons pass messages to each other
The simple message passing between neurons can lead to complex behaviours and functions
What is the role of axons and dendrites in neurons
Axons are responsible for transmitting electrical impulses and chemical signals away from the neuron’s cell body, while dendrites receive impulses from other neurons and transmit them towards the neuron’s cell body
How are neurons related to artificial neural networks
Artificial neural networks are inspired by the structure and function of biological neurons, and use mathematical models of neurons to perform tasks such as classification, prediction, and control
READ YOUR NOTES FOR IMAGES
NOW
