Chapter 1 Concepts plus regression review

Question 1

Three principles of design:

Answer 1

• replication
• randomization
• local control of error (blocking)

Question 2

Categories of experimental problems

Answer 2

• Treatment comparisons
• Variable screening
• Response surface exploration
• System optimization
• System robustness

Question 3

Steps to experiment planning

Answer 3

1. State objective
2. Choose response
3. Choose factor and levels
4. Choose experimental plan
5. Perform experiment
6. Analyze the data
7. Draw conclusions and make recommendations

Question 4

Experimental units vs observational units

Answer 4

• Experimental unit is item that is being experimented on and measured
• Observational unit can be thought of as a technical replicate

Question 5

Replication does the following:

Answer 5

• Replicating experimental units:
  
  • Allows estimation of the experimental error
  • Improves the precision of the estimates
• Replicating observational units:
  
  • mimizes the impact of measurement error
  • does NOT estimate experiment error

Question 6

Randomization does the following

Answer 6

• Distributes the impact of any systematic bias
  
  • ensures fair comparisons
  • if bias is present, inflatest the estimate of error
• Elimates presumption bias

Question 7

Local control of error does the following

Answer 7

• Also called blocking
• reduces the random error among the experimental units
• controls for anything which might affect the response other than the factors

Two important forms of local control of error:

• blocking
• covariates

Question 8

Block

Answer 8

Groups of homogeneous units

Question 9

Blocking

Answer 9

1. arranged so that within block variation is smaller than between block
2. should be applied to remove the block-to-block variation
3. randomization is applied to assignments of treatments within the blocks

Question 10

Derive the LSE estimators without matrix notation

Answer 10

Question 11

Derive the LSE estimators using matrix notation

Answer 11

(X’X)^-1X’y

Question 12

Derive the expectation, variance and covariance estimates of the LSE estimators without using matrix notation

Answer 12

Question 13

Derive the expectation, variance, and covariance of the LSE estimators using matrix notation

Answer 13

Question 14

How do you estimate σ² in LSE

Answer 14

σ_hat² =

MSE =

RSS/(N-k-1)=

SUM (y_i - y_i_hat)² / (N-k-1)

y^T(I-H)y / (N-k-1)

For SLR, N-k-1 = N-2

Question 15

degrees of freedom in multiple linear regression

Answer 15

df overall = N-1

df model = k

df error = N-k-1

Note: this assumes k regressors + intercept

such that model matrix is n x k+1

Question 16

standard error beta1_hat

Answer 16

=sqrt(MSE/Sxx)

Question 17

SST =

Answer 17

SST = SSReg + SSRes

Question 18

R² = (in SS components)

Answer 18

R² = SSreg/SST = 1 - SSres/SST

Question 19

R² = (Pearson format)

Answer 19

R² = SxySxy / SxxSyy

Question 20

ANOVA style F stat for MLE

Answer 20

F = (MSReg/df1) / (MSres/df2)

where df1 = difference in full and reduced model parameters

df2 = total df - number of parameters in full model

Question 21

Confidence interval for beta1_hat in SLR

Answer 21

beta1_hat +/- t_{n-2,alpha/2 * se(beta1_hat)}

Question 22

Confidence/prediction interval for y_hat( at x₀)

Answer 22

CI: y_hat_xo = t_N-2,alpha/2 * sqrt(MSres) * sqrt(1/N + (X_bar - Xo)²/Sxx)

PI: y_hat_xo = t_N-2,alpha/2 * sqrt(MSres) * sqrt(1+ 1/N + (X_bar - Xo)²/Sxx)

Question 23

Answer 24

Question 25

Extra sum of squares principle

Answer 25

• Two models: Model I (reduced) and Model II (full)
• Model I is a sub-model of Model II
• F stat = (SSred - SSfull)/(q-k) /
• (SSfull/df_full)

Question 26

R^2, R_a², and C_p (SS’s)

Answer 26

R² = 1 - SSres/SST

R_a² = 1 - MSreg / SST/df_total

C_p = SSres/MSE - (N-2p)

Question 27

AIC and BIC

Answer 27

AIC = N ln (SSres/N) + 2p

BIC = N ln (SSres/N) + p ln (N)

Question 28

Model selection methods

Answer 28

Forward selection

Backwards elemination

Stepwise

Best subsets

Question 29

E(C_p) =

Answer 29

If model is true, E(RSS) = (N-p)σ²

E(C_p) = (N-p)σ²/σ² - (N-2p)

=p

Question 30

MLR CI for beta_hat_j

Answer 30

beta_hat_j +/- t_{N-k-1, alpha/2} * sqrt ( MSE * (X’X)^-1_jj )

Question 31

MLR contrast

aTbeta ~ ( , ) and

test stat

Answer 31