Lecture 3 - Linear Regression Flashcards
What is Linear Regression?
It predicts a value by multiplying the input feature weight, summing them up and adding a bias - REFER TO SLIDES FOR THE FORMULA
Linear Regression Example
REFER TO ONENOTE
What is the Normal Vector and How is it used in the performance measure
A mathematical equation that gives the directly -> to find the value of theta that minimises the cost function (MSE) - REFER TO SLIDES FOR FORMULA
Linear Regression Code (With Normal Vector Code example)
REFER TO ONENOTE
What is Gradient Descent?
An Optimisation algorithm that is used to tweak parameters iteratively to minimise a cost function (MSE). It mesures the local gradient of the error function with regards to the parameter vecter theta and its goes in the direction of descending gradient. Once the gradient reaches zero its has reached a minimum.
Gradient Descent Example (CODE AND MATH)
REFER TO SLIDES
What are some issues with Gradient Descent?
When n (as in the greek letter) is too small there is a slow convergence
When n is too big it can overshoot, it never converges or it can even diverge
What are the 3 ways to do Gradient decent?
Batch GD
Stochastic GD
Mini-Batch GD
What is Batch GD?
Uses the whole training set to compute the gradients at every step, which make it very slow when the training set it large
What is Stochastic GD?
Just picks a random instance in the training set at every step and computes the gradients based only on that single instance. Makes the algorithm much faster since it has very little data to manipulate
Example of Batch GD
REFER TO SLIDES
Example of Stochastic GD
REFER TO SLIDES
What is Mini-Batch GD?
At each step, computes the gradients on small random sets of instances (mini-batches). Get a performance boost from hardware optimization of matrix operations (e.g., vectorisation, GPUs).
Example of Mini-Batch GD
REFER TO SLIDES
What is Polynomial Regression?
Similar to Linear Regression but use when data is not a straight line. It shows the relationship with higher degree terms
Example of Polynomial Repression
REFER TO SLIDES
What are Learning Curves?
Learning Curves are plots of the model’s performance on the training set and the validation set as a function of the training set size. To generate the plots, train the model several times on different sized subsets of the training set.
What is Logistic Regression
Logistic Regression (also called Logit Regression) is commonly used to estimate the probability that an instance belongs to a particular class (e.g., what is the probability that this email is spam?).
How does a Logistic Regression model make a prediction?
A Logistic Regression model computes a weighted sum of the input features (plus a bias term). The probability ˆp estimated by the Logistic Regression - REFER TO SLIDES FOR FORMULA
Logistic Regression Example
REFER TO SLIDES - LINK TO TRAINING AND COST FUNCTIONS
