Week 11: Core Skills In Regression

Question 1

What is regression known as?

Answer 1

Conditional Expectation Function = E(Y|X)

Question 2

What is a conditional expectation function?

Answer 2

It tells us the expected (predicted) value of Y for some set of X variables.

Question 3

Which variables do we include when using regression as a predictor?

Answer 3

All the variables, regardless of their statistical significance

Question 4

What is another main use of regression?

Answer 4

To find marginal effects

Question 5

Describe a marginal effect.

Answer 5

The impact of a one-unit change in X on E(Y |X).

Question 6

What is the marginal effect in linear regression?

Answer 6

The coefficient of the variables.

Question 7

What is differentiation used for in statistics?

Answer 7

• Compute marginal effects from regressions
• Find the minimum or maximum point of mathematical functions

Question 8

Define ‘estimand’.

Answer 8

The unknown parameter(s) that we aim to estimate [e.g. E(Y )]

Question 9

Define ‘estimator’.

Answer 9

Functions of sample data which we use to learn about
the estimands.
[e.g. the sample mean mean estimator 1n (sum of n (i) =1 y(i) ]

Question 10

Define ‘estimate’.

Answer 10

Particular values of estimators that are realised in a given sample dataset.
[e.g. the mean of a sample µ]

Question 11

What are the estimands in regression?

Answer 11

The βs, the true population coefficients.

Question 12

What are the estimators in regression?

Answer 12

The OLS regression function.

Question 13

What are the estimates in regression?

Answer 13

The βˆs, the estimated coefficients from our
regression.

Question 14

Why is there uncertainty in statistics?

Answer 14

Due to the process of sampling, we observe only one of many possible estimates from the full population. Our sample mean or regression coefficient is an imprecise estimate of the true population estimand.

Question 15

Why is the sampling distribution of the estimator important?

Answer 15

The sampling distribution of the estimator shows the probability of different estimates over repeated samples.

Question 16

List the 4 sampling distribution facts.

Answer 16

• Mean is the true β
• Normally distributed (Central Limit Theorem)
• Can estimate its variance from the sample variance
• The standard error is its standard deviation

Question 17

How do we create a sampling distribution in R?

Answer 17

By using simulation

Question 18

How do we approximate the sampling distribution?

Answer 18

By calculating the standard error (standard deviation).

Question 19

Define the sample variance.

Answer 19

An unbiased estimator of the variance of the true
sampling distribution of any β(j) from a multiple regression.

Question 20

When do we say β is statistically significant?

Answer 20

If, under the null hypothesis, it would only have occurred 5% of the time or less, e.g. when |t| > 1.96.

Question 21

What does the standard error say?

Answer 21

The bigger it is, the more uncertainty we have about the true value of β.

Question 22

When do we reject the null hypothesis?

Answer 22

When α = 0.05, reject the null hypothesis when |t| > 1.96, e.g. reject when p < 0.05 under the null hypothesis

Question 23

When do we accept the null hypothesis?

Answer 23

If the 95% confidence interval contains 0, we cannot reject the null hypothesis.

Question 24

What is the Pseudo-Bayesian Approach?

Answer 24

A different approach involves directly simulating the sampling distribution, and using it to quantify uncertainty unlike using the standard deviation.

Question 25

How would we take n draws in R?

Answer 25

rnorm(n,mean=,sd=)

Question 26

When is simulation most useful?

Answer 26

When we want to show uncertainty about a
function of the coefficients such as the predicted outcome for a given set of X variables.

Question 27

Write the 5 steps to using simulation.

Answer 27

1. Estimate a regression model
2. Create n simulations of the coefficients using the multivariate normal distribution [in R, use the sims() command in the arm package]
3. For each set of n simulated coefficients, calculate the function required, storing the results
4. The 95% confidence interval is the 0.025th and 0.975th values of the vector from (3) [95% of possible values are contained within it]
5. The standard error is the standard deviation of the vector from (3)

Question 28

What does the command %*% do in R?

Answer 28

Carries out matrix multiplication, e.g. (pred.outcomes <- values %*% t(coefs))

Question 29

What is coefs?

Answer 29

A matrix with 1000 rows and five columns (each row is a simulation).

Question 30

What is values?

Answer 30

A vector of X used for prediction: 1 row and five columns