SRM Chapter 2 Flashcards
1
Q
SLR
A
- Simple Linear Regression
- Relationship between two numeric variables
- Parametric
2
Q
MLR
A
- Multiple Linear Regression
- Multiple predictors (x’s) used to predict the dependent variable (y).
- Parametric
3
Q
Residuals
A
- e = y - y-hat
- For each i
- Want this to be minimized obviously
- This is done by: ordinary least squares tries to minimize the sum of the squared residuals.
4
Q
Partitioning of Variability
A
5
Q
Parameter Estimates
A
6
Q
R-squared
A
- Coefficient of determination
- Portion of variability in the response explained by the predictors
- R-squared = SSR/SST
- Between 0 and 1 (is a %).
- Want this to be high
7
Q
Adjusted R-Squared
A
- Adjustment for MLR that accounts for the number of predictors
- Does not have to range from 0 to 1.
8
Q
B0
A
- Intercept parameter
- Free parameter
- Equal to y when x is 0.
9
Q
B1
A
- Slope parameter
- Free parameter
- For every unit increase in x, y increases by B1 * x.
10
Q
SLR Model Assumptions (6)
A
- Yi - B0 + B1Xi + ei
(linear function plus error) - xi’s are non-random
- Expected value of ei is 0.
-> so expected value of Yi is B0 + B1Xi (ei cancels to 0). - Variance of ei is sigma-squared.
-> Because E[ei] = 0, the variance of Yi is also sigma-squared.
-> Also homoscedasticity (variance constant across all observations). - ei’s are independent (across observations (?)).
- ei’s are normally distributed (across observations (?)).
11
Q
Homoscedasticity
A
- Variance (sigma-squared) is constant across all observations
12
Q
b0
A
- Estimate of B0 to get y-hat
13
Q
b1
A
- Estimate of B1 to get y-hat
14
Q
Method to estimate b0 and b1
A
- Ordinary least squares/method of least squares
15
Q
Ordinary Least Squares
A
- Determines estimates b0, b1
- Optimization equation
- Estimators are unbiased (bias = 0).
16
Q
MSE
A
- Mean squared error
- Estimate of sigma-squared
- Denominator is n-2
- Unbiased, so bias is 0.
- Best fit is when MSE is minimized.
17
Q
RSE
A
- Residual standard error
- Aka residual standard deviation
- sqrt(MSE)
18
Q
Design Matrix
A
- X
- Matrix for the x’s (?)
19
Q
Hat Matrix
A
- H
- Aka projection matrix
- H times vector of actual responses = fitted values of response
- In other words, y-hat = H*y
20
Q
b Matrix
A
- 1*2 matrix of b0 and b1
21
Q
y Matrix
A
- Matrix for actual observed values of y
22
Q
SSR
A
- Regression sum of squares
- Proportion of variability in y explained by the predictors
23
Q
SSE
A
- Error sum of squares
- Aka sum of squared residuals
- Proportion of variability in y that cannot be explained by the predictors
24
Q
SST
A
- Total sum of squares
- Total variability (both explained and unexplained)
- SST = SSR + SSE
25
Q
Positive Residual
A
- Actual observation > (larger than) predicted observation
26
Q
Negative Residual
A
- Actual observation < (smaller than) predicted observation
27
Q
Null model
A
- Y = B0 + e
- No predictors (x’s)
- No relationship between y and x’s
28
Q
Do you want R-squared and Adjusted R-squared to be high or low?
A
- High
- Means more of the variance in y can be explained by the predictor(s).
- Want this to be as high as possible so that the unexplained variance is minimized.
29
Q
Is R-squared or Adjusted R-squared better for comparing MLR models? Why?
A
- Adjusted R-squared
- Because R-squared increased as predictors are added so a larger R-squared doesn’t necessarily mean a better model.
- But Adjusted R-squared accounts for the number of predictors so it is a better method of comparison between models.
30
Q
Two-tailed t Test (Hypothesis Test): What are we testing, and why?
A
- Test to see whether the slope parameter is 0 (B1 = 0).
- H0: B1 = 0
- H1: B1 <> 0
- If true, then there is no relationship between the x’s and y.
- So, we want to reject H0 to say that it’s plausible that there is a linear relationship between x’s and y.
31
Q
Test Decision (Two-Tailed t Test)
A
- For significance level a, reject H0 if:
- |t-stat| => ta/2,n-2
- p-value <= a
32
Q
One-Tailed t Test (Hypothesis Test): What are we testing and why?
A
- Same as two-tailed but sometimes it’s more appropriate to only have to reject one region
- Looking to prove that there is a positive slope between x and y.
33
Q
When do we use a right-tailed t test?
A
- When only a right tail rejection is needed
34
Q
When do we use a left-tailed t test?
A
- When only a left tail rejection is needed
35
Q
Confidence vs Prediction Interval
A
- Confidence: range for the mean response (across all observations)
- Prediction: range for the response of a new observation
- Prediction > Confidence (prediction is always at least as wide as the confidence interval).
36
Q
Confidence Interval
A
- Range that estimates the MEAN response
- Narrows in the middle
37
Q
Prediction Interval
A
- Range that estimates a NEW observation’s response
- Narrows in the middle (when the chosen predictor value is also the sample mean of the predictor)
38
Q
Why is the prediction interval at least as wide as the confidence interval?
A
- Prediction accounts for the variance in e in addition to Y-hat
- Have to cast a wider net to predict a new single response as opposed to the mean response over all observations
39
Q
Regression Coefficients
A
- B0, B1,…,Bp (Bj’s).
- B0 is still the intercept
- B1,…,Bp are regression coefficients instead of slope because that no longer makes sense with multiple predictors (x’s).
40
Q
Added assumption for MLR
A
- Predictor xj must not be a linear combination of other p predictors
- Because if an xj is a linear combination of other predictors it doesn’t add any additional information about the relationship between x’s and y.
41
Q
Nested Models
A
- Models that share a set of predictors
- Each model is a subset of the next model with more predictors
42
Q
Nested MLRs: p
A
- p is a measure of flexibility
