L6 - L2 Regression Flashcards
What is the nature of machine learning?
- nature just refers to what happens in the real world
- we have some exogenous variables (A), that are affected by nature that produce B quantities of interest
- e.g. the features of the house and their respective price
- How can we figure a forecast of what a price would be based off its features
- We need to build a model of reality (ML) –> we choose the variables
in theory, what are we doing in Machine Learning?
- Machine Learning does make alot of mistakes
- This is due to a lack of theory of my model explains things in the real world
- we just have real-world data and apply models to it
- This is due to a lack of theory of my model explains things in the real world
Based off nature what is the model we are creating in machine learning?
- its a model of reality/nature that is trying to explain how A relates to B
—There is a close connection between machine learning, optimization and statistics.
—What are the ingredients?
1) Data: A
2) The model: g(A) –> g is determined by any type of mathematical relationship ( in this module we will assume there is a linear relationship)
3) Loss function to be optimized L(A,g(A))
What is the notation we use for Machine Learning?
What is the Linear Model we will be using in this module?
Why do we put 1 in the final column of the matrix A?
this is artificially added to the matrix when we want to do the regression
What is the overfitting problem?
- When the model gets accustomed to old data –> gets exact equations to solve old data
- Thus is inference when using new data is wrong –> the model may not be completely correct looking at the universe of data as a whole
- GAUSS suggest that instead of finding the coefficients that fit the data perfectly that you should find values of the data that minimise the loss function –> minimises the mean squared errors
What is the expected loss function?
- W use the expectation as it is a function of random variables (B an A) and therefore it is a random loss
What are the ingredients we need for a loss function?
What is the Mean square error (quadratic loss) function?
What is the Expected Square loss minimisation problem in general?
What is the solution Gauss came to when minimising the expected square loss function?
How to find the analytical solution to a regression?
- write out the equations as bi - aixi for all rows in the matrix
- find partial derivatives of all xi
- simultaneously solve the equations for x
- put in the form b = a1x1 +a2x2…..
Optimal solution using the Gauss formula?
