Logistic Regression Flashcards
connections between glms and linear models
link function, iterative re-weighted least square, response function, weights, weighted residual sum of square, F- test of glms
https://www.sagepub.com/sites/default/files/upm-binaries/21121_Chapter_15.pdf
logistic regression: deal with perfectly separated data
maximum likelihood estimate is not well defined
assumption of logistic regression
log odd ratio between different class is a linear function of X
low correlations in X
observations independent with each other
how to estimate logistic regression parameters
Newton’s method (iterative re-weighted least square)
beta_new = beta_old - (Hessian)^-1 \partial(log likelihood)/\partial(beta_old)
estimate the variance-covariance matrix of logistic regression parameters (all glms)
use Fisher information, which is estimated as I(beta) = -Hessian. Var(beta) = I(beta)^-1
z test for logistic regression
Wald stastic hat(beta)/hat(se(beta)) ~ N(0, 1)
Use it as z-score to calculate the p-value
because the dispersion parameter for logistic and poisson models are fixed as 1. (variance and mean are correlated and no further estimate for variance is needed)
likelihood ratio test for logistic regression
LR = 2 (loglikelihood(model_large) - loglikelihood(model_small))
chi-square distribution with df=p_large - p_small
wald (chi-square) test for logistic regression
wald chi-square hat(beta)^2/hat(var(beta)) ~chi^2(1)
use a chi-square distribution to calculate the p-value
an approximation of likelihood ratio test
lagrange multiplier test (score test) for logistic regression
the gradient of the likelihood function—known as the score—evaluated at the hypothesized parameter value under the null hypothesis.
approximate chi-square distribution
roc curve
type-I error, type-II error
sensitivity, specificity
