Cross Sectional Flashcards
What are two requirements for linear regression
1) Additive errors 2) One multiplicative parameter per term
What are some important requirements for dummy variables?
They cannot depend on Y You cannot include intercept and all dummies. Would give perfect collinearity (Xs are not linearly independent)
Derive the OLS beta estimator
What is the formula for R^2 centered?
RSS/ TSS = 1 - SSE/TSS
What is TSS, RSS and SSE
When to use centered and uncentered R^2?
When model does not contains constant, use uncentered
What are the large-sample asymptotic assumptions of regression?
1) Stationary ergodicity 2) Full rank (E[Xi’Xi]) 3) Martingale difference sequence (error times regressor) 4) Moment existence. Regressors have finite 4th moments and errors have finite variance
What is estimator bias?
Difference between expected value of estimator and true value of parameter
What is estimator consistency?
And estimator is consistent if its probability limit is the true parameter
What is the problem with ommited variables?
If the estimated excludes relevant variables that are correlated with the included variables, the parameter estimates will be biased. Also, parameter variance will not be estimated consistently. It is safe to exclude dummy variables since they are orthogonal to regressors
What happens if you include extraneous (non-relevant variables)
You parameter estimates will still be unbiased and variance will still be consistent. But you will have more variance (potentially too low t-stat)
What are the problems of working with heteroscedastic data?
Parameters will be unbiased, but variance estimator will be inconsistent. One solution is to use White’s robust variance estimator. Using White’s estimator on homoscedastic data will however give worse finite sample properties and increases likelihood of size distortions. Another solution to heteroscedastcity is to use GLS
How can use test for heteroscedasticity?
Use White’s test. Estimate your model. Then regress squared errors on squares and cross products of all regressors (including constants). Null hypothesis: all parameters (except for intercept) are zero. Test statistic n*R^2. It is chi^2 squared distributed with df = number of regressors in auxiliary regression (excluding intercept)
What happens when errors are correlated with regressors?
If measuring regressors with noise. Regressors and errors may be correlated. This gives downwards bias. Endogeneity will also give bias.
What is a type 1 error
Rejecting a true null
What is a type 2 error?
Failure to reject the null when the alternative is true
What is the size of a test?
Pr(Type 1 error) = alpha
What is the power of a test?
Pr(1- type 2 error). Probability of rejecting a null when the opposite is true
What is a linear equiality hypothesis?
R*beta - r = 0 Where R is an mxk matrix r is mx1 vector m i number of restrictions k is number of regressors
What are the 3 types of hypothesis tests?
Wald test Lagrange Multiplier test Likelihood Ratio test
What are the differences in power between the 3 types of hypothesis tests?
W is about the same as LR which is larger than LM in terms of test statistics. Since they follow the same distribution, larger test statistics gives more power
How to implement a Wald test
Run the unrestricted regression and estimate parameters and covariances. Compute test statistics using null hypothesis. Compare against chi-squared distribution
How to implement a Lagrange Multiplier test
That is, the Lagrange Multiplier Test examines the size of the “shadow price” of the constraint regarding we are trying to test in the OLS optimization framework. Estimate the restricted model and compute its errors. Calculate the score using errors from the restricted model and the regressors from the unrestricted model Calculate the average score and the varianc of the score Compute the test statistic
How to implement a Likelihood ratio test
The Likelihood Ratio test is based on testing whether the difference of the scores, evaluated at the restricted and unrestricted parameters, is large Estimate the unrestricted model and its parameter covariance Compute the test-statistic using the restrictions Compare against CVs from Chi^2_m distribution
What are the main model selection techniques?
General to specific (GTS) Specific to general (STG) Information Criteria Cross-valiadation
Explain general to specific
Start with the largest possible number of possible models If at least one regressor is insignificant, remove the regressors with the lowest t-stat. Restimate the model. Continue until all regressors are significant
Explain specific to general
Start with the variable with the lowest p-value. At new variables sequentially as long as most recently added variable is significant
Discuss pros and cons of GTS and STG
GTS has a positive probability of including irrelevant variables, it will however never exclude relevant variables. In STG, the variance is NOT consistently estimated in the beginning, which can lead to wrong inference. For both, we have the problem that t-stats do not follow standard distributions when used sequentially.
Explain Information Criteria
If possible, search through all models and pick the one where the information criteria is lowest. Typical information criteria are AIC and BIC. If not feasible, run GTS or STG type search through models. Information criteria are of the form: - log-likelihood + penalty term for including regressors. In OLS use ln(error variance) instead of ll.
Discuss pros and cons of different information criteria
The AIC asymptotically may select a model with irrelevant regressors
Explain k-fold cross validation
Divide the data into k group. For each group, estimate beta excluding the data from the group. Calculate SSE error using the beta estimate on the data in the group. Sum SSE for all group. Select the model with lowest total SSE.
What are some important specification tests?
Chow test. For stability / structural breaks
RESET test: for misspecified model
Rollig/recursive parameters
Residual plots
What is a chow test?
it is a test for parameter stability. Regress y on xs and x*indicator(s) Null is that indicator-parameter is zero. Test using Wald, LR or LM
What is a RESET test?
Test for a misspecified model. Estimate the model. Calculate y_hat Run regression of y on original regressors, plut squared y_hat, cubed y_hat etc. Null: parameters for y_hat are zero. If rejected, some non-linear model may be better
How should outliers in regression by defined?
In terms of errors. Could upper an lower quantiles (e.g. 0.025). Defining errors in terms y would lead to downwards bias
Explain two methods of deadling with outliers?
Trimming. Removing outliers observations Winzorisations. Setting observations above threshold to threshold (in terms of errors) After this, rerun the regression
Explain residual bootstrap and when to use it
Estimate beta_hat and errors in full sample Resample errors and xs independetly. Compute y_tilde from resample x and errors and beta_hat Estiamte beta_tilde Repeat k times (e.g. 500) Take variance of beta-tildes Can only be used if data is homoscedastic
Explain non-parametric bootstrap
Resample x and y pairwise. Estimate beta_tilde. Repeat k times (e.g. 500). Take variance of beta_tildes
What is the test statistic in a Wald test?
Distributed Chi^2 m, where m is number of restrictions
What are some key factor that affect the power of a test?
Sample size
Distance between null and true value
Estimator precision
