stats-linear regression

Question 1

for statsmodel regression, a high p-value (eg, > 0.5) suggest?

Answer 1

the intercept coefficient is not significantly different from zero, the variable holds little predictive power

Question 2

for statsmodel regression, a low p-value (eg, < 0.05) suggest?

Answer 2

the intercept coefficient is significantly different from zero, the variable holds predictive power

Question 3

regression: sum of squares total

Answer 3

sum of ((dependent variable - dependent variable mean)^2)….. measures the total variability of the dataset (variance)

Question 4

regression: sum of squares regression

Answer 4

sum of ((predicted value - mean)^2)… measures the explained variability by the regression line

Question 5

regression: sum of squares error

Answer 5

sum of ((observed value - predicted value)^2)… measures the unexplained variability by the regression line. “error”

Question 6

regression: relationship among SST, SSE and SSR

Answer 6

SST = SSE + SSR. Total variability = explained variability + unexplained variability

Question 7

OLS

Answer 7

ordinary least square, aims to minimize SSE

Question 8

R-squared

Answer 8

SSR/SST: variability explained / total variability. in [0, 1]. A higher R-squared means a better regression model

Question 9

adjusted R-squared

Answer 9

always < R-squared because it penalizes excessive use of variables.

Question 10

when to drop an independent variable

Answer 10

When adjusted R-squared is lowered. F-statistics is lowered. The pvalue for that variable is high.

Question 11

F-statistics

Answer 11

testing overall significance of the model. A higher F-statistics means better model. prob(F-statistics) is the pvalue for the F-statistics, and should be close to zero for a good model. It tests the null hypothesis that betas = 0