Logistic Regression Continued Flashcards
Why would we include more than one variable in a model?
- Investigate multiple associations at once
- Adjust for confounding
- Model effect modification
- Increase the power of analysis of randomised trials
- Provide better predictions by accurately modelling data
When conducting regression with binary data what can be used?
Logistic regression
What does logistic regression involve?
Modelling the log odds (logit) of the probability of outcome
Why is the logit link used instead of the log odds?
The logit link is used so we can model probability on a linear scale;probability is bounded between 0 and 1
The log odds takes any number from -infinity to +infinity. It is unbounded
What do we measure the log odds with in a logistic regression?
A linear predictor
To interpret coefficients how must we exponentiate them?
exp(Bo) = eBo= odds of Y = 1 when x1= 0
if x1 is binary
exp(B1) = eB1= odds ratio comparing x1 = 1 when x1= 0
if x1 is continuous
exp(B1) = eB1= odds ratio comparing x1 = x+1 to x1=x
i.e odds ratio for a one unit increase in x1
What else needs to be exponentiated?
Confidence intervals
Lower limits(OR) = ell
Upper limits(OR) = eul
Standard errors are given on the log scale and thus do not need to calculate CIs from standard errors on the odds ratio scale
An odds ratio of 1 means no effect - CI significant if excludes 1
p-values need no exponentiating
When doing logistic regression what is assumed?
- Outcome is binary
- Observations are independent, conditional on covariates in the model
- The linear predictor is correctly specified
* The log odds are correctly modelled by additive combinations of the variables in the model
The linear predictor can contain as many variables as we need
True or false
True
log(p(y=1)/1-P(Y=1) = B0, B1x1, B2x2, B3x3, B4x4…
Exponentiating any coefficient gives what?
An odds ratio that can be interpreted as with the single variable model
What are coefficients now conditional upon?
Other coefficients in the model
How must categorical variables be handled and what is this sometimes called?
By splitting into multiple binary variables
Sometimes called one hot endcoding
There will be one less variable than categories, the category with no variable is the reference category
* E.g. race (coded as White, Black, Hispanic, Other)
Coefficients for categorical variables are relative to what?
Reference category
How must categorical variables be coded to be used in regression and what can they have?
Numerically and they can have value labels
Putting what in front of a variable in stata tells it to treat it as categorical?
What happens if this is not included?
an i.
Stata will treat the variable as continuous
In stata what is the default category?
The one with the lowest number
- The default category can be changed by having b# in front of the i
- They can have value labels
Eg. to have the category coded as 4 as reference
- What happens when a categorical variable is added to a model?
- What needs to be done consequentially?
You add more than one variable and will get more than one p-value reported
Correctly test whether there is an association between the categorical variable and the outcome a joint test must be carried out
Two types of test:
Wald test: Simpler to implement
- Generally give same result
Likelihood ratio test: Better statistical properties
- It is best practice to report the p-value from the joint rather than the p-values for individual categories
- Report confidence intervals and estimates for individual categories
What can a likelihood ratio
test be used to compare?
Nested models
- Model A is said to be nested in model B if model B contains at least all the variables that are in model A
E.g. a model adjusting for age and smoking is nested in a model adjusting for age, smoking and gender.
What does likelihood refer to?
How likely the data is to be observed based on the parameter estimates.
What do Likelihood ratio tests compare?
The likelihood from the two models
Fit both models
Compare the likelihood
What do Wald tests use?
A quadratic approximation to the likelihood to calculate p-values based on the fitted model only
The p-values you see in the output from linear regression are what?
Wald tests
Joint tests (Likelihood ratio or Wald tests) must be used when..
Testing associations of categorical variables
Odds ratios from multiple logistic regression are conditional on what?
Other variables in the model
What is the table 2 fallacy?
Reports of multiple adjusted effect estimates from a single model. This practice, which remains common in published literature, can be problematic when different types of effect estimates are presented together in a single table.
It is not obvious how to interpret coefficients
* There are different reasons why a variable may show no effect
* Keep in mind:
* Confounders
* Mediation
* Colliders
If a third variable is a common cause of both the variable of interest and the outcome it is called what?
A confounder
What happens if we do not adjust for the confounder?
The association between the variable and outcome will be distorted
What will adjusting for the confounder reveal?
The correct association
Confounders lie on the causal pathway between the variable of interest and outcome
True or false
FALSE
Confounders do not lie on the causal pathway between the variable of interest and outcome
If a variable lies on the causal pathway between variable of interest and outcome it is called what?
A mediator
What will adjusting for a mediator reduce?
The association between the variable of interest and outcome
If a variable is caused by both the outcome and exposure it is called what?
A collider
Adjusting for the collider distorts what?
The association between outcome and exposure
What is an example of a collider?
Hospital admission
Adjusting for hospital admission can make two
unrelated conditions appear associated
What is the guidance if interested in a single variable?
- Attempt to adjust for confounding
- Include Variables that you suspect might cause the variable of interest or the outcome
- Include potential confounders regardless of whether they are statistically significant
- Be careful adjusting for variables that could be caused by the variable of interest
- Mediators
- Colliders
No need to report coefficients from suspected confounders in model
What is the guidance if interested in multiple variables?
- Be mindful of possible causal structures
- Presence of mediators may mean that no association does not mean that variable is not a cause or would
not be associated if intermediate factor not adjusted for - Colliders can distort associations
- Be clear in interpretation that adjustment for other factors has occurred
What is the guidance if interested in prediction modelling?
Often appropriate to include all variables that may be predictive in model
* You do not necessarily need to report effects of components
* If you do avoid providing causal interpretation
Care must be taken when interpreting coefficients from what?
Multiple logistic regression
- When extending a linear model each term is multiplied by what?
- What does this encode?
- How do continuous variable act?
- What is the effect of variable?
- A beta parameter and added to the other
- assumption about the linear predictor
- “linearly”
- Additive
What do additive linear predictors assume?
The effect of a change in one variable is the same regardless of the value the other variables take.
What is effect modification?
A variable which changes the effect of a variable of interest and is not on the causal pathway between the variable of interest and the outcome
For example the effect of age on heart attack risk may be moderated by smoking status
When the effect of a variable in our model changes with another variable this is called what?
Effect modification.
What is Subgroup analysis in clinical trials is an example of?
Effect modification
Subgroup analysis aim to show what?
Whether treatment effects differs according patient characteristics
E.g. is treatment more effective for smokers than non-smokers
Here the effect is modified by smoking status
What do we model modification with?
An interaction term
Interactions are model terms where two variables are…
multiplied together
How can we carry out statistical tests for effect modifiers?
To test whether an effect modifier is statistically significant you must test whether the interaction term is significant
Comparing p-values at different levels of the moderator will lead to increased type 1 error rates as multiple tests are carried out. This leads to concluding there is moderation, when there is not.
When a categorical variable is involved in effect modification all levels must be tested simultaneously (likelihood ratio test, wald tests)
Report confidence intervals and estimates for each level of effect modifier
What are other uses for interaction terms?
Can be used to improve the fit of the model to the data
This may be useful when adjusting for confounding to capture true relationship between confounder and outcome (rarely used)
Also good for prediction when aim is to find model which best predicts unseen data
Excessive use of interaction or non-linear terms can lead to overfitting or challenges in interpretation
T
Typically we assume what?
Variables act additively in logistic regression
What can interactions be used to allow?
The effect of a variable to be different depending on the value of other variables in the model e.g effect modification
What is used in Stata to estimate effects when more than one variable is used to model as a prediction?
lincom
When using statistical tests to assess effect modification what should be
reported?
p-value for the interaction term should
What can be used to make predictions?
A logistic regression model
The prediction is the value of their linear predictor
- To get the odds of the person experiencing an event you exponentiate what?
- To get the probability you use what?
- The linear predictor
- The transformation
How can a model fit be compared and assessed?
AIC and BIC
Calibration
Discrimination
What does a lower AIC and BIC mean?
The model is good
Likelihood ratio tests can be used to compare nested models but what
if the models are not nested?
AIC (Akaike information criteria) and BIC (Bayesian information criteria) take into account the how well the model fits and the number of parameters
BIC penalizes more heavily for more parameters in model
Often the same conclusions are the same from both measures
What does calibration refer to?
The level of agreement betweenobserved and predicted values – if we predict a 20% probability of an event for a group of people the observed frequencies should beclose to 20%
How can calibration be assessed?
By statistical tests, plots or linear regression
What does a linear regression of observed values against predicted probabilities give?
A slope and intercept – these are sometime referred to as calibration in the small and calibration in the large
Calibration can also be stratified by what?
Covariates
What is a good way to evaluate calibration?
using plots comparing predicted probabilities to observed outcomes
One approach is to group observations by predicted probabilities, then plot predicted frequencies vs observed frequencies from each group
Alternatively the Lowess smoother can compare predicted probabilities to the outcome
Calibration plots should be compared to the line with slope 1 and intercept zero – this line represents perfect calibration
What statistical test can be carried out for calibration?
Hosmer-Lemshow test
Group observations into bins based on predicted values or covariate patterns, compare observed to predicted frequencies in the bins
Lower p-value = poorer fit: A statistically significant result indicates evidence that differences between observed and predicted frequencies were unlikely to happen by chance.
Can either select a number of bins e.g. 10 or if only categorical covariates have a bin for every pattern
What are drawbacks to the Hosmer-Lemeshow test?
p-value dependent on sample size – whether test is significant depends on goodness of fit and sample size
p-value can vary with number of bins – no clear way to decide
What does discrimination describe?
Ability of model predictions to differentiate between those with and without the outcome
What does good discrimination mean?
The predictions are higher for those with the outcome and lower for those without.
When does poor discrimination occur?
When there high and low predictions occur with similar frequencies for those with (or without) the outcome
What can discrimination be assessed using?
C-statistic
What is the c-statistics also known as?
Area under the curve (AUC)
What is the c-statistic?
C-statistic is the probability a randomly selected person who has a positive outcome has a higher predicted probability than a randomly selected patient who does not have a positive outcome.
How is the c-statistic calculated?
By comparing all possible pairs of patients with different outcomes and comparing predicted probabilities.
What does a c statistic of 0.5 mean?
No predictive ability
What does a c-statistic of 1 mean?
Perfect prediction
What is the AUC mathematically equivalent?
The definition of a c-statistic, they always agree
In an AUC the receiver operating curve compares what?
Sensitivity with specificity
Sensitivity: True positive rate - Correct prediction for someone who has the outcome
Specificity: True negative rate - Correct prediction for someone who does not have the outcome
How is AUC calculated?
By taking the area under the curve formed by varying a cutoff for classifying positives and negatives
Sensitivity is plotted against
1 - specificity
No predictive ability would result in the sensitivity being equal to
1– specificity
Perfect predictive ability based on AUC would mean what?
Sensitivity and specificity were always one
When using statistical tests to assess effect modification the p-value for the interaction term should be reported
TRUE or FALSE
True
