Quiz 1

Question 1

Linear model assumptions

Answer 1

(1) Linear in parameters
(2) Random sampling
(3) No perfect collinearity
(4) Zero conditional mean assumption: The mean of the population error is zero for all values of x
(5) Homoscedasticity: The variance of the population error is the same for all values of x
(6) Population error is independent of predictor variables and normally distributed with mean 0

Question 2

What if assumption 6 does not hold?

Answer 2

This is OK in large sample sizes due to the Central Limit Theorem

Question 3

What if assumption 5 does not hold?

Answer 3

Can use weighted least squares or robust methods, or try transformations of Y

Question 4

How to identify homoscedasticity

Answer 4

Plot residuals against predicted or predictor values, or use the Breusch-Pagan test.

Question 5

Total sum of squares

Answer 5

sum(y_i - y bar)^2

Question 6

Sum of squares explained

Answer 6

sum(y hat - y bar)^2

Question 7

Sum of squares residuals

Answer 7

sum(y_i - y hat)^2

Question 8

R^2 (formula and interpretation)

Answer 8

R^2 = SSE/SST and is the coefficient of determination, representing the proportion of variance explained by the model

Question 9

Goodness of fit evaluation for logistic regression

Answer 9

Compare observed vs expected outcomes for each covariate pattern and use Pearson’s X^2 test. If there are many covariate patterns, use deciles of risk (Hosmer-Lemenshow). You can think of the test statistic as a sum of square residuals. Also, you can plot the residuals vs observations to identify outliers.

Question 10

Concerns with Hosmer-Lemenshow technique

Answer 10

Don’t choose G too small, need large data sets, and with very large data sets you may get a large C despite good fit (so check the table)

Question 11

What do we need to consider about B_0 hat and B_1 hat?

Answer 11

They may not be independent; may need to consider their covariance

Question 12

What three properties should estimators have?

Answer 12

Unbiased, meaning the expected value is equal to the population value for any population value
Consistent, meaning they converge in probability to the population value as sample size grows without bound
Efficient, meaning it has the lowest variance of all possible estimators

Question 13

Method of moments: general concept

Answer 13

Replace expected values with their sample means

Question 14

Gauss-Markov Theorem

Answer 14

If assumptions 1-5 hold, the OLS estimator is BLUE

Question 15

Which assumptions are necessary for inference using the OLS estimator?

Answer 15

Requires assumptions 1-6

Question 16

What is the method of moments estimator for the variabce, and why?

Answer 16

We use (1/(n-p-1) sum(eta_i hat)^2 because otherwise there will be bias due to the degrees of freedom issue.

Question 17

What can you do if the zero conditional mean assumption does not hold?

Answer 17

Nothing. You are screwed

Question 18

What do you need to do if testing combinations of the beta hats?

Answer 18

Need to account for covariance between the beta hats

Question 19

When can you use a robust method of inference, and what is the general idea?

Answer 19

In the presence of homoscedasticity, and when the sample size is sufficiently large. This method estimates the variance of beta hat based on the fitted residuals, resulting in downweighting of any outliers.

Question 20

When can you use weighted least squares, and what is the general idea?

Answer 20

In the presence of homoscedasticity. Weights observations by the reciprocal of their variance. Can develop weights by assuming a linear relationship between x and variance, or using regression.

Question 21

What increases the variance of a beta estimate?

Answer 21

Multicollinearity: correlation between predictors in the model
X values concentrated together
Variance of the outcome variable

Question 22

What happens to a beta estimate when you omit a variable?

Answer 22

The initial beta estimate becomes equal to the estimate for that variable, plus the estimate for the other variable times the correlation between the two variables.

Question 23

Adjusted R^2: purpose and formula

Answer 23

Way to compare models with different number of covariates. Not so precise. R^2_adj = 1-(SSR/(n-p-1))/(SST/(n-1))

Question 24

Residual mean squared: purpose and formula

Answer 24

Deciding how many covariates to put in the model. Graphing the average residual mean squared for various models with each possible number of covariates, looking for a plateau. (Sometimes the model with the smallest residual mean squared might not have that number of covariates, however.)

Question 25

Model comparison options

Answer 25

Adjusted R^2
Residual mean squared
Mallow's Cp
AIC and corrected AIC
Cross-validation

Question 26

Mallow’s Cp: purpose and formula

Answer 26

Similar to residual mean squared technique. Specify a maximal model and compare it to a nodel with p parameters, using the formula Cp = SSRp/(theta hat)^2_max - (n-2p). If p parameters is enough, then E(Cp) = p. Can graph Cp vs number of covariates.

Question 27

Leave one out cross-validation: purpose and formula.

Answer 27

Determine if model is overfitted by using it for predictions. Sum the predicted residual sum of squares for each model, and the model with the lowest PRESS is best. May not be feasible in very large data sets.

Question 28

Types of linear transformations

Answer 28

Scaling, centering, standardization

Question 29

Disadvantages of categorization of variables

Answer 29

Not sure if the categories are “right”, eats df, can only easily compare to the single reference group, may have residual confounding

Question 30

Possible nonlinear transformations

Answer 30

log, polynomial

Question 31

Disadvantages of polynomial approach

Answer 31

May not fit well at the extremes; not sensitive to local nonlinearities; can be affected by outliers

Question 32

Idea behind penalized spline regression

Answer 32

Instead of choosing number of knots, keep in all the knots and restrict sum of squared betas for spline part be less than or equal to a constant C. This penalizes roughness (can adjust based on smoothing parameter lambda, which is = 0 using all knots).

Question 33

Goal of quadratic splines

Answer 33

First derivatives equal at knots; gives smoother fits

Question 34

Ridge and lasso regression concepts

Answer 34

Ridge: Constrains coefficient sums of squares to fixed value
Lasso: Same as ridge but can reduce some coefficients to zero

Question 35

How do you get the variance in maximum likelihood estimation

Answer 35

In large samples, use the observed information matrix (parameter estimate variance is asymptotically normal)

Question 36

Inferences in logistic regression

Answer 36

Wald-based inferences based on z statistic – the beta estimator is asymptotically normal

Question 37

How many fisher scoring iterations should you have

Answer 37

A small number

Question 38

How do you get the variance of a beta coefficient estimate in logistic regression

Answer 38

Look at the observed information matrix in the (k,k)th element

Question 39

Likelihood ratio test: goal and formula

Answer 39

G=-2log(likelihood nested model/likelihood larger model) ~ X^2 _q-p. Tests whether parameter(s) are statistically significant.

Question 40

Deviance: formula, simpler formula, and corollary in linear reg

Answer 40

-2log(likelihood fitted model/likelihood saturated model). If y values are 0/1, simplifies to -2log(likelihood fitted model). Equivalent to SSR in linear regression.

Question 41

AIC formula for logistic regression

Answer 41

AIC = -2log(likelihood fitted) + 2(p+1), or D+2(p+1) for 0/1 models

Question 42

Goal of exact logistic regression

Answer 42

Allows inferences for logistic regression in small sample sizes

Question 43

Probit regression

Answer 43

Similar to logistic regression but with a link function using the cumulative normal distribution

Question 44

Conditional logistic regression

Answer 44

Used in case-control/paired studies because number of parameters can get very large. Instead, we treat each pair as a unit of observation, knowing that each pair has a Yi=1 and a Yi=0.