Relationships between variables 1 - Logistic Regression

Question 1

Define Sensitivity

Answer 1

The proportion of individuals with the disease who are correctly diagnosed by the test.

The percentage of cases that had the observed characteristic (e.g., “yes” for obese) which were correctly predicted by the model (i.e., true positives).

Question 2

Sensitivity calculation

Answer 2

Sensitivity: true positives / true positives + false negatives

Question 3

State reasons why researchers investigate relationships between variables

Answer 3

Discover new associations. / Explain variance. / Compare relative influence of factors on outcomes. / Adjust analyses for potential confounders. /Develop prediction models.

Question 4

Describe the purpose, use, strengths and limitations of scatterplots

Answer 4

Scatterplots - Test bivariate relationships
Purpose: visual representation of the relationship between two quantitative variables
Use: Identify patterns, trends and potential outliers
Strengths: Simple, intuitive, and provides a clear visual indication of relationships.
Limitations: Does not provide a quantitative measure of the strength or direction of the relationship.

Question 5

Describe the purpose, use, strengths and limitations of correlation

Answer 5

Purpose: Quantify the strength and direction of the relationship between two variables.
Use: Calculate correlation coefficient (Pearson’s r for linear relationships).
Strengths: Provides a single number that summarizes the relationship; easy to interpret.
Limitations: Only measures linear relationships; sensitive to outliers; does not imply causation.

Question 6

Describe the purpose, use, strengths and limitations of regression

Answer 6

Purpose: Model the relationship between one dependent variable and one or more independent variables.
Use: Predict values, assess the impact of variables, and understand relationships.
Strengths: Provides a detailed understanding of relationships; can handle multiple predictors; provides estimates of effect size.
Limitations: Assumes a specific form (linear, logistic, etc.); can be sensitive to outliers; requires more complex interpretation.

Question 7

Correlation vs Regression

Answer 7

Correlation summarises the strength and direction of relationship between two variables as a single value (r correlation coefficient).

Whereas Regression is a model that uses one variable as a predictor (x) and the other as a response (y) to model the relationship.

Regression finds an equation that best describes the relationship between the two variables.

While regression allows one variable to be predicted from the other, Correlation doesn’t allow predictions of one variable from the other.

The null hypothesis of correlation is ‘there is no (linear) relationship between variables’.

Whereas the null hypothesis of regression is ‘coefficients associated with variables = 0’.

Question 8

State the assumptions of linear regression

Answer 8

Dependent variable/response - Continuous
Independent variable/predictor - Continuous or categorical
Relationships between variables - Bivariate relationship between predictor and response variables is linear
Residuals - the residuals of the regression line are approximately normally distributed
Normality - No outliers in data

Question 9

State the assumptions of logistic regression

Answer 9

Dependent variable/response - Dichotomous (two outcomes) / binary or ordinal (ordered categories)
Independent variable/predictor - Continuous or nominal (mutually exclusive categories)
Relationships between variables - There shouldn’t be multicollinearity (several independent variables are correlated)
Residuals - N/A

Question 10

Define Specificity

Answer 10

The percentage of cases that did not have the observed characteristic (e.g., “no” for not obese) and were also correctly predicted as not having the observed characteristic (i.e., true negatives).

Question 11

Specificity calculation

Answer 11

Specificity: false positives / false positive + true negatives

Question 12

Compare and contrast between correlation and regression

Answer 12

Correlation:

• Summarises strength and direction of relationship between 2 variables as a single value
  r = correlation coefficient

Correlation does not allow for the prediction of one variable from the other

Null hypothesis = No linear relationship between variables

Regression:

-Model
-Uses one variable as the predictor (x) and the other as the response (y)

Regression allows one variable to be predicted from the other

Null hypothesis = coefficients associated with variables = 0

Question 13

What is linear regression

Answer 13

linear regression is a statistical model which estimates the linear relationship between a scalar response and one or more explanatory variables.

Fits a straight line to the data

y = a + bx + e

y = outcome variable
x = predictor variable

a and b are model parameters:

b is the slope of the line
a is the intercept of the line on the y-axis
where x = 0
e = ‘residual error’

Question 14

State the assumptions for logistic regression.
Dependent variable/response
Independent variable/predictor
Relationships between variables
Residuals
Normality

Answer 14

Dependent variable/response - Dichotomous (two outcomes) / binary or ordinal (ordered categories)
Independent variable/predictor - Continuous or nominal (mutually exclusive categories)
Relationships between variables - There shouldn’t be multicollinearity (several independent variables are correlated)
Residuals - N/A
Normality - Not considered

Question 15

State the assumptions for linear regression
Dependent variable/response
Independent variable/predictor
Relationships between variables
Residuals
Normality

Answer 15

Dependent variable/response - Continuous
Independent variable/predictor - Continuous or categorical
Relationships between variables - Bivariate relationship between predictor and response variables is linear
Residuals - the residuals of the regression line are approximately normally distributed
Normality - No outliers in data

Question 16

Explain why researchers investigate relationships between variables.

Answer 16

Discover new associations. / Explain variance. / Compare relative influence of factors on outcomes. / Adjust analyses for potential confounders. /Develop prediction models.

Question 17

The theory underpinning logistic regression

Answer 17

Predicting true or false: e.g., obese or not obese (measures categories)
Sigmoid curve shape – curve will be shifted to find the best fit using thresholds.
Want to see how the data fits onto a line to determine how good the model is good at predicting; this is done through a log odds scale. Log(probability) /log(1-probability).
Rotate until you have the best line of fit – SPSS.

Question 18

Define Risk

Answer 18

Absolute risk: probability of an event occurring in a sample/population.
Number of people with event / total number of people

Question 19

Define Odds

Answer 19

Chance of an event occurring vs not occurring in a population.
Number of people with event/number of people without event

Question 20

What is a ROC Curve

Answer 20

A receiver operating characteristic curve (ROC), is a graphical plot that illustrates the performance of a binary classifier model at varying threshold values. The ROC curve is the plot of the true positive rate against the false positive rate at each threshold setting.

(ROC curve: plot of binary classifiers, predicting classification using a threshold).

Question 21

Describe situations where investigators may wish to investigate relationships between multiple variables

Answer 21

To explore what contributes to an event, such as disease or death.

Question 22

the theory underpinning logistic regression

Answer 22

Logistic Function: The logistic regression model uses the logistic function to model the probability of the dependent variable. The logistic function is an S-shaped curve that maps any real-valued number into the (0, 1) interval, making it suitable for binary outcomes.
Odds and Log-Odds: Logistic regression models the log-odds (logarithm of the odds) of the probability of the dependent event occurring. This transforms the relationship into a linear one that can be estimated using standard regression techniques.
Maximum Likelihood Estimation (MLE): Logistic regression typically uses MLE to estimate the model parameters. MLE finds the parameter values that maximize the likelihood of observing the given data.