Lecture 5 - Linear Regression

Question 1

Regression Analysis

Answer 1

Fit a relationship between a numerical outcome variable and a set of predictors

Question 2

Variables

Answer 2

• Numerical outcome variable Y also called response, target, or dependent variable
• Set of predictors X1, X2, …, Xn also referred to as independent variables, input variables, regressors, or covariates

Question 3

Linear Regression Model

Answer 3

Arranged in, or extending, along a straight or nearly straight line

Question 4

Single vs Multiple Linear Regression Model

Answer 4

Single: One independent prediction, i.e., single variable X

Multiple: two or more predictors, i.e., X1, X2, …

Question 5

Intuition for Single Linear Regression

Answer 5

Question 6

Intuition for Multiple Linear Regression

Answer 6

Question 7

Linear Regression Model

Answer 7

Question 8

Graphical visualisation of Linear Regression

Answer 8

Question 9

Ordinary Least Squares (OLS)

Answer 9

• Method for estimating the unknown parameters in a linear regression model
• It minimises the errors associated with predicting values for the dependent variable Y
  
  • It uses a least squares criterion because without square we would allow positive and negative deviations from the model to cancel each other out

Question 10

Ordinary Least Square (OLS) pt2

Answer 10

Question 11

Objectives for single/multiple regression

Answer 11

1. Predictive - detect the outcome value for new records given their input values
   
   1. Explanatory (or descriptive) - Quantifying / explaining the avg effect of inputs on an outcome & Data are treated as a random sample from a larger population of interest

Question 12

Explanatory objective - in single/multiple regression

Answer 12

Generate statements useful for decision making

E.g. a unit increase in X is associated with an average increase of 2 points in Y

Question 13

Predictive objective - in single/multiple regression

Answer 13

Given an ew value for X, estimate the value for Y

Question 14

Overfitting

Answer 14

• *Issue:**
• The outcome correspond exactly, or is extremely close, to the given data set
• I.e., the model learns the existing data too well
• *Consequences:**
• Model fails to include additional data or
• Generates unreliable predictions
• *Example:**
• Considering creating a model for predicting grades of students given the hours they study
• Using the data from UVT, applying it to TU/e data fails due to overfitting

Question 15

Underfitting

Answer 15

• *Issue:**
• Model can’t accurately capture the data dependencies
• Fails to identify effects supported by the data
• Usually this happens due to the model’s simplicity
• *Consequences:**
• Model has bad generalisation capabilities when applied with new data

Question 16

Complete Linear Regression Python example

Answer 16

Question 17

Quiz 1

Answer 17

Question 18

Quiz 2

Answer 18

Question 19

Quiz 3

Answer 19