Generalised Linear Models

Question 1

GLM Components

Answer 1

1. Random part
2. Systematic part
3. Link function

Question 2

Random Part

Definition

Answer 2

f(y;θ,φ) = exp[ (yθ-b(θ)) / φ + c(y,φ) ]

θ = canonical (or natural) parameter
φ > 0 , the scale parameter

Question 3

Systematic Part

Definition

Answer 3

-the linear predictor:
η = Σ βj xj
-sum from j=1 to j=p where p is the number of observations

Question 4

Link Function

Definition

Answer 4

η = g(μ) where μ=E(y)
=>
μ = g^(-1) (η) = h(η)
-where g is the link function and h is the inverse link function

Question 5

How to find the random part?

Answer 5

• write out the probability distribution function,f
• rewrite in terms of an exponential
• use the form of the random part to find θ, φ, b and c

Question 6

GLM

Expectation

Answer 6

E(y) = b’(θ)

Question 7

GLM

Variance

Answer 7

Var(y) = φ b’‘(θ)

Question 8

GLM

Expectation Proof

Answer 8

-by definition of the probability density function:
1 = ∫ f(yi,θ) dy
-differentiate wrt θ
-sub in f for a GLM to do the differentiation
-revert back to f
-separate into two integrals
-sub in E(y) from the definition of expectation into the first
-use 1 = ∫ f(yi,θ) dy in the second

Question 9

GLM

Variance Proof

Answer 9

-by definition of the probability density function:
1 = ∫ f(yi,θ) dy
-differentiate wrt θ
-sub in f for a GLM to do the differentiation
-revert back to f
-differentiate wrt θ a second time
-sub in definition of Var(y) and 1 = ∫ f(yi,θ) dy

Question 10

Systematic Part in Matrix Form

Answer 10

η = Σ βj xj
-but for each observation, yi, the explanatory variables may differ:
ηi = Σ βj xij
-or in matrix notation:
η = X β
-where η = (η1,…,ηn)^T is a vector of linear predictor variables, β = (β1,…,βp)^T is a vector of regression parameters and X is an nxp design matrix

Question 11

How to find the canonical link function?

Answer 11

-the canonical link function is a mathematically and computationally convenient choice of link function
-set θ=η
-since E(y) = b’(θ) = μ
=>
θ = b’^(-1) (μ) = η

Question 12

Canonical Link Function, g’(μ)

Answer 12

g’(μ) = 1/b’‘(θ)

Question 13

Maximum Likelihood Estimation for GLMs

Answer 13

f(y;θ,φ) = exp[ (yθ-b(θ)) / φ + c(y,φ) ]
-then the likelihood is:
L[f(y;θ,φ)] = Π f(yi;θ,φ)
-take natural log:
l[f(y;θ,φ)] = Σ [ yiθ-b(θ) / φ  +  c(yi,φ) ]
= n * [θy^ - b(θ) / φ  +  const.
-maximise wrt θ
-where y^ is the mean

Question 14

The Saturated Model

Definition

Answer 14

-when the data are fitted exactly to the model

μi^ = b’(θi^) = yi

Question 15

Deviance

Definition

Answer 15

D = 2φ [ l(θ~,y,φ) - l(θ^,y,φ) ]
-where the first likelihood term l(θ~,y,φ) refers to the saturated model and the second, l(θ^,y,φ), refers to the fitted model

Question 16

Deviance

Steps to Find

Answer 16

• express f(y) as an exponential
• log-likelihood is ln(f(yi)) summed over i
• find μi = b’(θi) to express parameters in terms of the mean estimate
• for the fitted model term, replace parameters with mean
• do the same for the saturated model term only with μi=yi
• sub into formula for D
• cancel any remaining terms that can be cacelled

Question 17

Deviance

Distribution, φ=1 - goodness of fit

Answer 17

-goodness of fit

D ~ χ²_(n-r)

Question 18

Deviance

Distribution, φ=1 - model comparison

Answer 18

-model comparison of M1 and M2
M1 = 1 + E
M2 = M2 + E + F
-i.e. model two considers one extra explanatory variable / effect
(D1 - D2) ~ χ²_{(r2-r1)

Question 19

Deviance

Distribution, φ unknown

Answer 19

-estimate φ with model 3:
φ = D3 / (n-r3)
-then test M1 against M2 with:
[ (D1-D2)/(r2-r1) ] / φ ~ F_(r2-r1, n-r3)

Question 20

Raw (response) Residuals

Answer 20

ei = yi - μi^

Question 21

Standardised Residuals

Answer 21

ei = (yi-μi^) / √b’‘(θi)

Question 22

Deviance Residuals

Answer 22

ei = sign(yi-μi^) √di

-where di is the deviance for individual i