Linear Models

Question 1

Recall the standard linear regression model, define any terms and indices

Answer 1

y_i=B₀+B₁x_i1+B₂x_i2+…+B_kx_ik+e_i

y_i is the ith response

xij is the ith value of the jth regressor

k is the number of regressors (k+1=p parameters)

B₀ is the y-intercept

B_j is coefficient associated with the jth regressor

e_i is an error term, usually assumed iid random with mean 0 and variance sig^2

Question 2

Matrix form of regression model and what are any assumptions made (OLS assumptions)?

Answer 2

y = XB + e

E[e]=0

var(e)=sig² I

Question 3

OLS estimate of B variance and expected value

Answer 3

Bhat = (X’X)^-1X’Y

Var(Bhat)= sig²(X’X)^-1

E(Bhat)=B

Question 4

What does OLS estimate, Bhat, do?

Answer 4

Minimizes the sum of squared residuals (in matrix form: SS_res= [y-yhat]’ [y-yhat])

Question 5

SS_res in quadratic form (Y’AY where A is symmetric)

Answer 5

Y’[I - X(X’X)^-1X’]Y

Question 6

Global F-test

Answer 6

H₀: B₁=B₂=…=B_k=0

H_a: B_j ≠ 0

Test statistic is MS_reg / MS_res ~ F

Where MS_reg = SS_reg / k

SS_reg = SUM (yhat_i - ybar)²

Question 7

Write SS_reg in terms of matrix notation

Answer 7

SS_reg = Y’ [X(X’X)^-1X’ - 1(1’1)^-11’]Y

Question 8

Describe a cell means model

Answer 8

y_ij= u_i + e_ij

y_ij is the jth observation for the ith group

i=1,2,…,t

j=1,2,…,n_i

u_i is the true mean for the ith group

e_ij is the error term

Question 9

For a cell means model, define y, M, and u

Answer 9

Question 10

What is the incidence matrix and what is uhat, M’M, (M’M)^-1?

Answer 10

Question 11

Less than Full Rank ANOVA Model (overly paramterized)

Answer 11

Where the row length of X is larger than the column length

Question 12

Difference between overly parameterized and fully parameterized

Answer 12

Overly paramterized models have columns in the X matrix which are linear combinations of the first column (ie not orthogonal)

Question 13

For less than full rank anova model, what happens to Bhat?

Answer 13

Since Bhat= (X’X)^-1X’Y it cannot be estimated because X’X does not have an inverse. But, X’X does have generalized inverses which have to be solved that way

Question 14

Consequences of Less than Full rank ANOVA models

Answer 14

Generalized inverses are not unique

There are no unique estimates for B

Serious limitations on what we can estimate and test

Usual focus: contrasts (pairwise comparisons, factorial effects)

Things of critical importance: eigenvalues and rank of matrices (provides degrees of freedom)

Question 15

If A and B are nxn square matrices then det[AB]=

Answer 15

det[A]det[B}

Question 16

If A is nxn then det[A]=0 iff

Answer 16

A is singular

Question 17

The rank of A is…

Answer 17

the greatest number of linearly independent columns (or rows) of A. (Linear dependence implies that at least one column (or row) of A can be written as a linear combination of the other columns (or rows) )

Question 18

If A and B are non-singular, then for any matix C

C, AC, CB, ACB

Answer 18

all have the same rank

Question 19

If A is an mxn matrix of rank r, then there exist non-singular matrices P and Q such that PAQ is one of the following:

Answer 19

Question 20

The rank of AB cannot

Answer 20

exceed the rank of either A or B

Question 21

If A is a nxn matrix then det[A]=0 iff

Answer 21

the rank of A is less than n

Question 22

The matrix of a quadratic form can always

Answer 22

be chosen to be symmetric

Question 23

A and B are said to be congruent matrices iff

Answer 23

there exists a non-singular matrix, C, such that

B=C’AC

We say C is the congruent transformation of A

Question 24

Let A be an nxn symmetric matrix of rank r. There exists a non-singular matrix C such that

Answer 24

C’AC=D

where D is a diagonal matrix with exactly r non-sero diagonal elements

Question 25

If A and B are congruent matrices, then

Answer 25

they have the same rank

Question 26

Let C be an mxn matrix with rank r then the ranks of C’C and CC’

Answer 26

are also r

Question 27

Let A be an nxn matrix. There will always exist

Answer 27

n eigenvalues that satisfy

det[A-LI]=0

where L is a diagonal matrix of the n eigenvalues

Question 28

Let A be an nxn symmetric matrix. The rank of A

Answer 28

is the number of non-sero eigenvalues

Question 29

Let A be an nxn matrix. A has at least one zero eigenvalue iff

Answer 29

A is singular

Question 30

Let A be an nxn matrix. The determinant of A is the

Answer 30

product of eigenvalues

Question 31

Let A be an nxn matrix and let C be any nxn non-singular matrix then

A, C^-1AC, CAC^-1

Answer 31

all have the same number of eigenvalues

Question 32

Let A be an nxn real matrix. A necessary and sufficient condition that there exist a nonzero y that satisfies Ay=ey

Answer 32

is that e be an eigenvalue of A

Question 33

Let P be an nxn matrix. P is called orthonormal iff

Answer 33

P^-1=P’ and therefore PP’=P’P=I

Question 34

Let A be an nxn matrix, and let P be an nxn orthonormal matrix, then

Answer 34

det[A]=det[P’AP]

Question 35

Let x and y be nx1 vectors. x and y are called orthogonal if

Answer 35

x’y=0 or y’x=0

Question 36

Let A be a mxn matrix and B be an nxp matrix. A and B are said to be orthogonal if

Answer 36

AB=0

Question 37

Let A be nxn symmetric matrix. There exists an orthonormal matrix P such that

Answer 37

P’AP=D where D is a diagonal matrix whose diagonal elements are the eigenvalues of A

Question 38

Let A be an mxn matrix with rank r>0. There exist matrices A_L (mxr with rank r) and A_R (rxn with rank r) such that

Answer 38

A=A_LA_R

Question 39

Define column and row space

Answer 39

Column space of a matrix A is the set of vectors that can be generated as linear combinations of the columns of A.

Row space of a matrix A is the set of vectors that can be generated as linear combinations of the rows of A.

Question 40

Let A be an nxn matrix. A is called positive semidefinite if

Answer 40

a) A=A’ (A is symmetric)
b) y’Ay ≥ 0 for all y
c) There exists at least one y ≠ 0 such that y’Ay=0

Question 41

If A, an nxn matrix, is positive semidefinite then (3 things)

Answer 41

1) The rank of A is less than n
2) The eigenvalues of A are greater than or equal to 0
3) If P is an nxn non-singular matrix then P’AP is also positive semidefinite

Question 42

Let A be an nxn matrix. A is positive definite if

Answer 42

a) A=A’ (A is symmetric)
b) y’Ay > 0 for all y≠0

Question 43

If A is positive definite then (3 things)

Answer 43

1) The rank of A is n
2) All of the eigenvalues of A are greater than 0
3) Let P be an nxn non-singular matrix. P’AP is also positive definite

Question 44

A matrix is called non-negative definite if it is

Answer 44

either positive definite or positive semidefinite

Question 45

Let C be an mxn matrix with rank r. C’C and CC’ are

Answer 45

both non-negative definite.

C’C or CC’ are positive definite iff they have full rank

Question 46

Let A be an nxn symmetric non-negative definite matrix. There exists some nxn matrix B such that

Answer 46

B’B=A

Question 47

Let A and B be nxn symmetric matrices. If A is positive definite, then there exists a non-singular matrix Q such that Q’AQ= ____ and Q’BQ=____

Answer 47

Q’AQ=I and Q’BQ=D where D is a diagonal matrix whose diagonal elements are the roots of det[B-lambdaA]

Question 48

If A and B are both non-negative definite, then there exists a matrix Q such that both Q’AQ and Q’BQ are

Answer 48

diagonal

Question 49

Let A be a nxn non-singular matrix partitioned into

A=[A₁₁ A₁₂

A₂₁ A₂₂]

where both A₁₁ and A₂₂ are square and non-singular and let A^-1=C

What is C?

Answer 49

Question 50

Answer 50

Question 51

Let A be an nxnx matrix. We call A idempotent if

Answer 51

AA=A

Question 52

If A is an nxn idempotent matrix with rank n, then A=

Answer 52

I

Question 53

If A is an nxn idempotent matrix of rank less than n, then A is

Answer 53

positive semidefinite

Question 54

If A is an idempotent matrix with rank r, then it has

Answer 54

r non-zero eigenvalues, each equal to 1

Question 55

Let A be an nxn (symmetric) idempotent matrix then

a) A’ is
b) Let P be an orthonormal matrix. P’AP is
c) Let P be an nxn non-singular matrix. PAP^-1 is
d) I-A is

Answer 55

a) (symmetric) idempotent matrix
b) (symmetric) idempotent matrix.
c) idempotent
d) (symmetric) idempotent matrix

Question 56

The trace of an nxn matrix A is the

Answer 56

sum of the diagonal elements

Question 57

Let A be an mxn matrix and let B be an nxm matrix.

trace[AB]=

Answer 57

trace[BA]

Question 58

If A, B, C are conformable, then

trace[ABC]=

Answer 58

trace[CAB] = trace[BAC] = trace[BCA] = trace[CBA] = trace[ACB]

Question 59

Let A be an nxn matrix and let P be a non-singular nxn matrix.

trace[A] =

if P is orthonormal then

trace[A]

Answer 59

trace[P^-1AP]

trace[P’AP]

Question 60

Let A be an nxn matrix with eigenvalues lam₁, lam₂, lam₃, … , lam_n

trace[A] =

Answer 60

SUM lam_i

Question 61

Let A and B be nxn matrices and let a and b be scalars.

trace[aA+bB] =

Answer 61

a trace[A] + b trace[B]

Question 62

Let A be an nxn matrix what does this say about the trace?

Answer 62

trace[A’] = trace[A]

Question 63

Show the general form to obtain a symmetric matrix

Answer 63

Question 64

A Moore-Penrose Inverse A⁺ for an mxn matrix A satisfies the following 4 conditions

Answer 64

1) AA⁺ is symmetric
2) A⁺A is symmetric
3) AA⁺A=A
4) A⁺AA⁺=A⁺

Question 65

Let X be an nxp matrix with rank p. Then X⁺=

Answer 65

(X’X)^-1X’

Question 66

Let A be an mxn matrix. Then every matrix A has a Moore-Penrose inverse, and ______

Answer 66

it is unique

Question 67

(A’)⁺=

Answer 67

(A⁺)’

Question 68

The Moore-Penrose inverse of A⁺ is

Answer 68

A

Question 69

If the rank of A is r, then each of the following matrices also has rank r (in terms of Moore-Penroses)

Answer 69

A⁺

AA⁺

A⁺A

Question 70

If A is non-singular, then A⁺=

Answer 70

A^-1

Question 71

If A is symmetric idempotent, then A⁺=

Answer 71

A

Question 72

The matrices AA⁺, A⁺A, I-AA⁺, I-A⁺A are all

Answer 72

symmetric idempotent

Question 73

For any matrix A, (A’A)⁺=

Answer 73

A⁺(A’)⁺

Question 74

Let P be an mxm orthonormal matrix. Let Q be an nxn orthonormal matrix, and let A be any mxn matrix. Then (PAQ)⁺=

Answer 74

Q’A⁺P’

Question 75

Let X=[X₁ X₂]

Then XX⁺X₁=

Answer 75

X₁

Question 76

Let A be an mxn matrix. A^- is called a generalized inverse of A if

Answer 76

AA^-A=A

Note, the Moore-Penrose inverse is also a generalized inverse

Question 77

Let X be an mxn matrix with rank r > 0. The following conditions hold for the generalized inverse

Answer 77

Question 78

What can be said about X(X’X)^-X’

Answer 78

It is invariant to the choice of generalized inverse

Question 79

X(X’X)^-X’=

Answer 79

XX⁺

Question 80

The following conditions hold for K=X(X’X)^-X’

Answer 80

Question 81

If A is an nxn non-singular matrix, then the system Ax=y

Answer 81

has a unique solution

Question 82

The system Ax=y has a solution iff (2 answers)

Answer 82

y is in the column space of A

AA^-y=y

Question 83

Let A be an mxn matrix. The system Ax=0 has a solution other than x=0 iff

Answer 83

the rank of A < n

Question 84

Let a and x be nx1 vectors. What are the derivatives with respect to x of

a’x, x’a, and x’x

Answer 84

a, a, and 2x

Question 85

Let A be an mxn matrix of constants and let x be an nx1 vector. Then the derivative of Ax with respect to x is

Answer 85

A

Question 86

What is the derivative with respect to x of x’Ax?

Answer 86

2Ax

Question 87

What is a stationary point, x₀?

Answer 87

x₀ is a solution to df(x)/dx = 0

Could be a min response, max response, or a saddle point

Question 88

What is the Hessian matrix and what information does it provide?

Answer 88

It is a matrix of second derivatives. It describes the funciton in the neighborhood of the stationary point.

Question 89

Conditions for Ordinary Least Squares estimation of B

Answer 89

Question 90

How to decompose variance-covariance matrix (where it is known that the matrix is symmetric)

Answer 90

Question 91

What does the generalized least squares estimate of B do?

Answer 91

Question 92

What is the GLS estimate of B?

Answer 92

Question 93

What is var(a’y) and var(Ay)?

Answer 93

Question 94

What is E[y’Ay]?

Answer 94

Question 95

Answer 95

Question 96

What are possible approaches toward minimizing the variance matrix of Bhat? (sig²(X’X)^-1)

Answer 96

1) min |sig²(X’X)^-1| which minimizes the volume of the confidence ellipsoid of the estimated coefficients
2) min tr[sig²(X’X)^-1] which would minimize the sum of the variances of the estimated coefficients

Both put an emphasis on eigenvalues

Question 97

Steps to show that an estimator is the best, linear, unbiased estimator

Answer 97

Question 98

Describe the density function for y, a px1 vector from the multivariate normal distribution

Answer 98

Question 99

If z is a nx1 univariate standard normal random variable then what can be said about z’z?

Answer 99

It is distributed chi-squared with n degrees of freedom

Question 100

What is the non-centrality parameter?

Answer 100

Question 101

Three things to consider if trying to show y’Ay follows a chi-squared and how to define non-centrality parameter

Answer 101

Question 102

Answer 102

Question 103

What test statistic should be used if Σ is known?

What test statistic should be used if σ² is unknown but V is known, where σ²V=Σ?

Answer 103

Chi-squared

F-distribution

Question 104

Answer 104

Question 105

The classic ANOVA table

Answer 105

Source df MS E[MS] F lambda

Question 106

Answer 106

Question 107

X_c‘1=

var(e)=

Σ=

Σ^-1=

Answer 107

Question 108

Define the cell means model

Answer 108

Question 109

Global F-test for cell means model

Answer 109

Question 110

ANOVA for cell means global F-test

Answer 110

Question 111

When using a contrast C for multiple comparison testing, what does the rank of C control?

Answer 111

The type I error rate for the set of comparisons

Question 112

Describe an identifiable parameter

Answer 112

Question 113

Describe an estimable funciton

Answer 113

Question 114

Answer 114

Question 115

Answer 115

Question 116

Let A be an mxn matrix. The system Ax=0 has a solution other than x=0 iff

Answer 116

the rank of A<n>
</n>

Question 117

Results of this

Answer 117

Question 118

All contrasts are _____

Answer 118

estimable functions

Question 119

What are the differences between pairwise comparisons for the cell means model and the effects model?

Answer 119

Question 120

For the generalized inverse case, test statistic given L’Bhat for an effects model

Answer 120

Where whatever is being tested is in the row space of X (L’=AX)

Question 121

Reasons the effects model is not very valuable from an analysis perspective

Answer 121

Resquires constraintes to be identifiable

Constraints ultimately must be linear combinations of the predicted values

Analysis must focus on estimable functions (contrasts are always estimable functions)

However, the effects model does generalize very nicely

Question 122

RBD Model and X matrix

Answer 122

X=[1 T M]

Question 123

Sets of Contrasts for RBD (to test treatments and blocks)

Answer 123

Question 124

Way to make orthogonality for X’X RBD

Answer 124

Question 125

ANOVA table for RBD.

Is there an appropriate test for fixed block effects?

Answer 125

Question 126

RCB model where interaction is important

Answer 126

Question 127

AB interaction contrasts for RBD interaction model

Answer 127

Question 128

Creating orthogonality for RBD interaction model

Answer 128

Question 129

ANOVA for RBD interaction model

Answer 129

Question 130

Interaction effect for RBD model and significance in relation to contrasts

Answer 130

If AB is significant, contrasts must focus on the comparing cell means

If AB is not significant, then contrasts would focus on the marginal means

Question 131

Model for one way random effects and variance of Y

Answer 131

Question 132

Main thing to test for random effects model

Answer 132

Question 133

ANOVA for one way random effects model

Answer 133

Question 134

For two way random effects model define distributions for A, B, AB

Answer 134

Question 135

ANOVA two way random effects model

Answer 135

Question 136

Distribution for SS of A, B, AB for mixed model

Answer 136

Question 137

ANOVA mixed model

Answer 137