Midterm

Question 1

Independence

Answer 1

If Pr(A & B) = Pr(A)*Pr(B) OR Pr(A | B) = Pr(A)
- Determine if variables are correlated
- Avoid biased estimates
- Investigate autocorrelation, which could lead to inefficiency and incorrect standard errors

Question 2

Conditional Expectation

Answer 2

• Calculating conditional Variance
• Law of iterated expectations

Question 3

Law of Iterated Expectations

Answer 3

E[E[g(x,y)|x1, x2]|x1] = E[g(x,y)|x1]
- Essential for deriving unbiased estimators
- How information affects predictions

Question 4

Mean Independence

Answer 4

E[Y|X]=E[Y]
- E[u|X]=0
- Helps to justify assumptions in instrumental variable regression

Question 5

Coefficient of Determination (R^2)

Answer 5

= SST / SSE
The proportion of the variance in the dependent variable that is captured by the model.
- Predictive Power
- Comparing models

Question 5

Leverage Point

Answer 5

Observation whose explanatory variables have the potential to exert an unusually strong effect on the fitted model.
- Can disproportionately influence the results of a regression model.
- Model stability
- Important to detect to ensure robust and reliable regression analysis

Question 6

Frisch-Waugh Theorem

Answer 6

Simplifies a MLR model by allowing it to focus on the impact of a single variable, holding others constant.

Beta_1 = (X1’ M2 X1)^-1 X1’ M2 y

• Useful when dealing with panel data sets and individual effects.
• How do individual variables contribute to the regression model?

Question 6

Positive Definite

Answer 6

If x’Ax >0 for all x != 0
- Ensures OLS estimates have a unique and stable solution
- Guarantees the Var-Cov matrix is invertible
- Ensures MLE reach a unique minimum

Question 7

Nonnegative Definite

Answer 7

If x’Ax >= 0 for all x != 0
- Var-Cov matrix must be non negative to make meaningful statistical inferences

Question 8

Square Root Matrix

Answer 8

R is a square root matrix if A = RR’
- Used in Monte Carlo
- Essential for Generalized Least Squares

Question 9

Level Curves

Answer 9

Sets of points where a function takes the same value.
c in R{x: x’Ax = c}
- Visualizing constraints and trade-offs

Question 10

Normal Distribution

Answer 10

f(y) = 1 / [2 sqrt(2pi o^2)] exp(- (y - mu)^2/(2o^2))
- Central Limit Theorem
- Standard statistical tests
- Symmetry and known properties make it easy to deal with

Question 11

Multivariate Normal Distribution

Answer 11

If it can be expressed as a non-stochastic linear transformation of a vector of independent standard normal random variables.
- Modeling relationships between multiple dependent variables (time series)
- Enables factor analysis and multivariate hypothesis testing

Question 12

Student’s t-Distribution

Answer 12

t(m) = z / sqrt(chi^2(m) / m)
- As sample size increases, it approaches the normal distribution
- Useful for small and large datasets
- Central to testing the significance of individual coefficients in regression analysis

Question 13

Snedecor’s F-Distribution

Answer 13

If the ratio can be expressed as the ratio of two independent chi^2 random variables each divided by its degrees of freedom.
- Comparing variances across groups
- Testing the significance of a regression model
- Test restrictions on multiple regression coefficients
- Testing if nested models provide a significantly worse fit than a more complex model

Question 14

Neyman-Pearson Lemma

Answer 14

The critical region should not contain any outcomes that are relatively more likely under the null than under the alternative.
- Likelihood Ratio Tests
- Balances the tradeoff between Type I and Type II errors

Question 15

Unbiased Test

Answer 15

The probability of rejection under each point in the null is always less than the probability of rejection under each point in the alternative.
- Provides a fair test that does not favour the null or alternative hypothesis.

Question 16

Restricted Least Squares Estimator

Answer 16

Incorporates constraints into the regression model.
- Testing hypotheses about relationships between variables
- Improves the efficiency of estimates

Question 17

Consistent Estimator

Answer 17

If the estimator converges in probability to the true value of the parameter.
- Essential for reliable long-term inference
- With sufficient data, the estimator will yield results close to the actual population parameter.

Question 18

Law of Large Numbers

Answer 18

Sample averages converge to their population means.
- Provides the foundation for the consistency of estimators.

Question 19

Asymptotics

Answer 19

The behaviour of estimators and test statistics as the sample size approaches infinity.

Question 20

Independent and Identically Distributed

Answer 20

Random variables that are independent of each other and drawn from the same probability distribution.
- OLS, MLE, LLN
- Simplifies assumptions -> Makes it easier to derive consistent and unbiased estimators.

Question 21

Continuous Mapping Theorem

Answer 21

plim g(x) = g(plim x)
- Allows the application of continuous functions to converging sequences of random variables while preserving convergence

Question 22

Convergence in Distribution

Answer 22

Let z be a sequence of random variables distributed by Fn. If Fn -> F, then we say z converges in distribution to F
- Analyzing the behaviour of r.v.’s as the sample size increases.
- Ensures the distribution of an estimator or test statistic approaches a limiting distribution
- Fundamental for Central Limit Theorem.

Question 23

Wald Test

Answer 23

Testing the significance of one or more coefficients in a regression model.
- Widely used in Generalized Linear Models and MLE

Question 24

Maximum Likelihood Estimator

Answer 24

The value of the parameter vector that maximizes the Likelihood Ratio Test.
- Produces efficient and consistent estimators.
- Central to Generalized Linear Models
- Incorporates non-linear relationships

Question 25

Efficient Score Test (the Lagrange Multiplier)

Answer 25

Tells us how the estimation changes if we relax the constraint.
- Useful when the unrestricted model is complex and difficult to estimate
- Effective for detecting small deviations from the null hypothesis

Question 26

Information Criteria

Answer 26

Balances model fit with model complexity
- Choosing the best model
- Time series

Question 27

Akaike (AIC)

Answer 27

Penalizes the inclusion of additional parameters
- Prevents overfitting
- Time series

Question 28

Bayes/Schwartz BIC

Answer 28

Imposes a stronger penalty for the number of parameters than AIC
- More conservative than AIC

Question 29

Rolling Binned Means Estimator

Answer 29

• Smooths time series data by calculating averages over sliding windows
• Reduces noise
• Mitigates the effects of outliers

Question 30

Kernel

Answer 30

Weigh observations based on their distance from a target point.
- Bias-variance trade off

Question 31

Curse of Dimensionality

Answer 31

The exponential increase in complexity and data requirements as the number of variables in a model grows

Question 32

Ridge Regression

Answer 32

Adds a penalty proportional to X’X which helps to shrink the coefficients towards zero without making them zero.
- Used to address multicollinearity
- Balances bias and variance
- Prevents overfitting

Question 33

Best Subset Selection

Answer 33

Identifies the best combination of predictors from a larger set of potential explanatory variables.
- Related to Information Criteria

Question 34

LASSO

Answer 34

Some coefficients are set to zero, while others are shrunk towards the origin.
- Improves model prediction and interpretability
- Balances model complexity and predictive accuracy

Question 35

Training and Validation Set

Answer 35

Sample is randomly divided into two parts: A training set and a validation set. Fit the model with the training set and use the validation set to compute MSE.
- Tests a model on unseen data.

Question 36

Leave one out Cross Validation

Answer 36

Train the model on all but one observation. Test it on the remaining observation. Repeat for each observation. Average the results.
- Unbiased estimate of model performance.
- Useful for small datasets.

Question 37

K-Fold Cross Validation

Answer 37

Divide into K groups. Treat each fold as the validation sample and average the MSE across the K groups.
- Works well for small and large datasets.
- More robust performance estimate

Question 38

Sieve Estimation

Answer 38

A linear combination of a family of functions that can be used to approximate m(x) arbitrarily well.
- Uses a sequence of simpler models instead of a fixed functional form.
- Avoids overfitting
- As the number of observations grows, the sieve converges to the true model.

Question 39

Neural Nets

Answer 39

Machine Learning Model
- Used for modeling complex non-linear relationships
- Flexibility and High Predictive Power

Question 40

Panel Data Fixed Effects

Answer 40

Controls for unobserved heterogeneity when this heterogeneity is constant over time and correlated with independent variables.
- Controls for time invariant factors