Generalised Linear Models Flashcards
A member of exponential family can be written in form
Θ in exponential family
Natural or canonical parameter
Φ in exponential family
Nuisance parameter (if unknown)
Score of exponential family
Hessian of exponential family
Fisher information matrix of exponential family
Expectation of score of exponential family
E[Y] for exponential family? Why?
Variance of score of exponential family
Variance of Y for exponential family
μ for exponential family
b’(θ)
Variance function for exponential family
a(φ) in exponential family
= (σ^2)/w where σ^2 is called the dispersion/scale parameter and w the weight
For normal distribution;
Θ =
μ
For normal distribution;
b(θ)
(Θ^2)/2
For normal distribution;
a(φ)
σ^2
For normal distribution;
c(y,φ)
For normal distribution;
E(Y)
For normal distribution;
Var(Y)
For normal distribution;
V(μ)
1
For poison distribution;
Distribution?
For normal distribution;
Distribution?
For poison distribution;
Θ?
log(λ)
For poison distribution;
b(θ)?
exp(θ)
For poison distribution;
a(φ)?
1
For poison distribution;
c(y, φ)?
-log(y!)
For poison distribution;
E(Y)
For poison distribution;
Var(Y)
For poison distribution;
V(μ)
μ
For Bernoulli distribution;
Distribution
For Bernoulli distribution;
Θ
log(p/(1-p))
For Bernoulli distribution;
b(θ)
log(1+exp(θ))
For Bernoulli distribution;
a(φ)
1
For Bernoulli distribution;
c(y, φ)
0
For Bernoulli distribution;
E(Y)
For Bernoulli distribution;
Var(Y)
For Bernoulli distribution;
V(μ)
μ(1-μ)
For binomial distribution;
Distribution
For binomial distribution;
Θ
log(p/(1-p))
For binomial distribution;
b(θ)
log(1+exp(θ))
For binomial distribution;
a(φ)
1/n
For binomial distribution;
c(y, φ)
For binomial distribution;
E(Y)
For binomial distribution;
Var(Y)
For binomial distribution;
Var(μ)
μ(1-μ)
Random component of general linear model;
Parameters
For each observation, given the fitted distribution, functions a,b and c (and usually) scale parameter φ are the same for all observations, only θ changes
Random component of general linear model;
Joint density
Random component of general linear model;
Vector y , observed responses
Is likelihood function for θ and φ
Systematic/Structural component of general linear model;
Linear predictor
Distribution of response
Systematic/Structural component of general linear model;
Design matrix
Link function does?
Describes relationships between E(Y) and linear predictor
Link function must
Any function g that is one to one, monotonic and differentiable (limitations May apply due to distribution) (eg poisson must have μ_i >0)
How to pick link function
-normally choose so that range is entire real line
Get θ_i from generalised linear model given that
Canonical link function
Canonical link function normal
Canonical link function poisson
Canonical link function Bernoulli/binomial
Normal linear model; linear predictor
Normal linear model; link between
Through the
Objective of binary regression
Model success probability p as a function of the covariates
Binary regression;
Θ when using canonical link
(Logit)
CDF of logistic dist
Binary regression;
Probit link
Using CDF of standard normal to model p(x)
Binary regression;
Probit link has g(μ)=
Where Φ is CDF of standard normal dist
Binary regression;
CDF of log-Weibull
Verify that log-Weibull CDF does in fact define a CDF
Binary regression;
Difference between logit, Probit and log-log link
Logit and Probit CDFs are symmetric about 1/2. Log-log link isn’t, hence this should be used when asymmetry as a function of the linear predictor is suspected
When to use Logit
Heavier tailed than standard normal dist, hence use when outliers are suspected in linear predictor
