Normal Linear Models

Question 1

Linear Regression

y

Answer 1

• dependent/response variable

- assume y is normally distributed for linear regression

Question 2

Linear Regression

x

Answer 2

x = (x1,…,xp)

• where p is the number of covariates
• independent variables / covariates / predictor variables

Question 3

Linear Regression

Model

Answer 3

y = α + Σ βj xj

-sum from j = 1 to j=p, where p is the number of predictor variables

Question 4

Linear Regression

Residual Sum of Squares

Answer 4

R = Σ (yi - μi^)²

• sum from i = 1 to n where n is the number of observations
• and μi^ is the fitted value for yi

Question 5

Linear Regression

Compare Models With F-Statistic

Answer 5

-compare model 0 with model 1
-null hypothesis: model 0 is best
-alternative: model 1 is best
F01 = (Ro-R1)/(r0-r1) / R1/r1
-where:
r = residual degrees of freedom
R = residual sum of squares

Question 6

Logistic Function

Answer 6

logistic(x) = 1 / (1 + exp(-x))

Question 7

Logit Function

Answer 7

-inverse of the logistic function

logit(q) = log(q / 1-q)

Question 8

Types of Variable

Answer 8

• quantitative

- qualitative

Question 9

Types of Variable

Quantitative

Answer 9

• continuous

- count

Question 10

Types of Variable

Qualitative

Answer 10

• un-ordered categorical
• -dichotomous (two categories)
• -polytomous (more than two categories)

-ordered categorical

Question 11

Types of Normal Linear Model

Quantitative Explanatory Variable, p=1

Answer 11

-simple linear regression

y = α + βx1 + ε

Question 12

Types of Normal Linear Model

Quantitative Explanatory Variable, p>1

Answer 12

-multiple linear regression
y = α + Σ βixi + ε
-sum from i=1 to i=p

Question 13

Types of Normal Linear Model

Dichotomous Explanatory Variable, p=1

Answer 13

-two sample t-test
-dichotomous: x=1 or 2
y = α + γ I(x=2) + ε
-where, I(x=j) = { 1 if x=j, or 0 else}

Question 14

Types of Normal Linear Model

Polytomous Explanatory Variable, p=1

Answer 14

-one-way anova
-polytomous: x=1,…,k
y = α + Σ δj I(x=j) + ε
-sum from j=1 to j=k

Question 15

Matrix Representations of Normal Linear Models

Answer 15

Y = XΒ + E
-where Y is an nx1 vector of observations, X is an nxp ‘design matrix’, B is a px1 vector of parameters and E is an nx1 vector of errors

Question 16

Constructing the Design Matrix

Answer 16

1) first column is a vector of 1s (intercept)
2) for each explanatory variable:
- -if quantitative: add xi as a column
- -if qualitative: add k dummy columns taking values 0 or 1, then remove one of these columns
3) for interaction terms e.g. for x1*x2, add a column of x1 values multiplied by x2 values

Question 17

Notation for Models

~

Answer 17

~ = modelled by / regressed by

Question 18

Maximum Likelihoos Estimation

Answer 18

• the likelihood is equal to the probability density function, f(y)
• take ln to get log-likelihood
• differentiate with respect to parameter and set equal to zero

Question 19

Normal Distribution

Probability Density Function

Answer 19

f(y) = 1/σ√2π * exp{-1/2 * [(x-μ)/σ]²}

Question 20

Poisson Distribution

Probability Mass Function

Answer 20

f(y) = λ^y * exp(-λ) / k!

Question 21

Binomial Distribution

Probability Mass Function

Answer 21

f(y) = mCy * p^y * [1-p]^(m-y)

Question 22

What is the purpose of a generalised linear model?

Answer 22

-to model the dependence of a dependent variable, y, on a set of p explanatory variables, x=(x1,…,xp), where ,conditionally on x, observation y has a distribution which is not necessarily normal

Question 23

Normal Linear Model

Definition

Answer 23

-a model that assumes the distribution of the dependent variable is Gaussian