Data Interpretation Flashcards
What should be included in a good figure caption
Should be able to interpret the results just by looking at the figure, caption, legends without having to read the text associated with it
Brief description for the treatment conditions
Brief description of results
Statistical tests
Number of data points used to create graphs
What error bars and points represent
What is the difference between descriptive and inferential statistics
Descriptive
- describes and summarises a data set
- calculations are made without uncertainty
Inferential
- inference about a parameter of interest in the population, based on what is observed in a sample
- calculations are estimated with a degree of uncertainty
What are the 2 forms of inferential statistics
-what is the difference between them
Estimation
-estimation of a population parameter of interest from the value observed in the sample
Hypothesis testing
-way to test differences in the parameter of interest between groups and produces a p value
What are the 2 forms of estimation
Point estimate
-single value (best guess) of the parameter in the population
Interval
- defined by 2 numbers between which the population parameter is said to lie
- examples include confidence intervals
What are the types of variable
-what are examples of each type
Categorical Qualitative -classified in categories without intrinsic ordering Binary (2 categories) - sex Nominal (2+) - ethnicity
Ordinal (order values from low to high) - age groups
- spacing between values does not have to be consistent
- age groups
Numeric
Quantitative
Discrete - cell counts
Continuous - height
What operators would be used for these variables
- qualitative
- ordinal
- quantitative
Qualitative - = or ≠
Ordinal - < or >
Quantitative - +, =, x
How would you work out the average and spread for these variables
- qualitative
- ordinal
- quantitative
Qualitative
- mode
- frequency distribution - frequency tables, graphs (histograms, pie chart)
Ordinal
- median - 50% percentile
- absolute ranges - range between max and min value
- percentiles - the value below which a certain percent of observations fall
- IQR - the range between the 25th and 75th percentile
Quantitative
- mean
- variance - average squared deviation of each observation from the mean
- SDs - square root of the variance
How does are the mean and median influenced by extreme values
Means is affected by extremes
Medians are not
In the situation where you have extremes, medians would be a better measure of averages
Describe what the mode, median and mean would be in a normal distribution
Bell curve
All 3 would be the same
Describe what % of observations would fall within -1SD -2SD -3SD in a normal distribution
1SD - 68% of observations would be within 1SD from the mean
2SD - 95% of observations would be within 2SD from the mean
3SD - basically all observations would be within 3SD from the mean
What are the 2 possible skewed distribution curves
-how will this affect the mean and median
Negatively skewed - the longer tail of distribution points in a negative direction
-mean is less than the median
Positively skewed - the longer tail of distribution points in a positive direction
-mean is more than the median
What are the dangers associated with categorising continuous variables
But why is this done
Done to improve clinical interpretation of results
However this may lead to
- a loss of information, leading to a loss of statistical power (loss of ability to detect a difference)
- the impact of the choice of cut-offs on results is problematic when the choice is not based on a strong a priori rationale
What is the difference between a population vs sample
-when is the use of a sample ok
Population
-represents whole group we are interested in
Sample
- too time consuming and expensive to contact the whole population
- representative group taken from population
- use this sample to infer information about the whole population
Sample results are appropriate when they are
- valid - sample is representative of population
- accurate - sample size is large enough
How do standard error, standard deviation and confidence interval differ from each other
Standard error
- describes the accuracy of the point estimate
- used to calculate CI
- 95% confidence interval indicates the range of values likely to include the true value in the population
Standard deviation
- measure of spread (variability)
- used for descriptive statistics to calculate intervals showing variability in the data
- for data sets with a normal distribution, 95% of data points will fall within 2SDs of the mean
Confidence interval
-estimated range of values likely to include the true unknown value in the population
Why do we use 95% confidence intervals
-what does this mean
Good compromise between 90% and 99%
95% CI
-if we repeated the same sampling from the same population 100 times, 95 of the 100 CIs would contain the true population parameter
How would we calculate the standard error of the mean
SD of sample/square root of sample size
How would you calculate the 95% CI of the mean
-what are we assuming with this calculation
sample mean +- (1.96 x SE)
We expect 95% of sample means to lie within (1.96 x SE)
We assume that
- the mean of all samples that could be drawn from our population follow a normal distribution
- SE corresponds to the SD of all sample means (this is different from the SD of the original data)
- distribution follows the 3sigma rule
How does sample size impact on the accuracy of your estimate
SE is inversely proportional to the square root of the sample size
So larger the sample size, the greater the accuracy, lower SE and narrower CI
How to calculate SE for binary variables
-when would you use this
square root [p(1-p)/n]
p = sample proportion n = sample size
SE of proportion is an approximation and only of use if the sample size is large
-a large sample size allows us to assume that the estimate is from a normal distribution and the SE is well estimated
np and n(1-p) should exceed 5 for the SE to be a good approximation
How would you calculate the 95% CI of the proportion
p +- (1.96 x SE)
What is a
- hypothesis
- hypothesis testing
Hypothesis - a statement about the true value of parameters and the relationship in a defined population
Hypothesis testing - procedure, based on the observed values of the parameters in a sample of the population, to determine whether the hypothesis is a reasonable statement
What are the steps involved in hypothesis
Define hypothesis
Perform test
Calculate test statistics
-the measure that summarises the difference or relationship that you want to test
Estimate p value
-tells you if you should accept or reject your H0
Interpret test results
What is the difference between the
- null hypothesis H0
- alternative hypothesis H1
H0
- assumed to be true
- there is no true difference or relationship between the observed values in the sampled population
H1
-there is a true difference between the observed values in the sampled population
What is the difference between a 2 sided and 1 sided alternative hypothesis
2 sided
- the difference can be in either direction
- default
1 sided
- the difference can be in 1 direction only
- recommended if there is strong supporting evidence that the effect can be in one direction only
For qualitative tests, which statistical test would you use
- unpaired
- paired
Unpaired - Pearson X2
Paired - McNemar X2
For ordinal tests, which statistical test would you use
- unpaired
- paired
Unpaired - Mann Whitney U
Paired - Sign
For quantitative unpaired tests, which statistical test would you use
- parametric
- non-parametric
Parametric - Student’s T test
Non-parametric - Mann Whitney U
For quantitative paired tests, which statistical test would you use
- parametric
- non parametric
Parametric - Student’s T test
Non-parametric - Wilcoxon signed-rank test
What does the
-p value
-a
tell you
p value
-The probability of getting the result in our sample by pure chance when H0 is true
a
- arbitrary cut off to help us evaluate the size of p
- the probability of committing a type 1 error
- the 5% chance of rejecting the H0 incorrectly
Important to note that statistically significant results are not always clinically significant. Clinically significant results are not always statistically significant
How would you interpret the p value
- U0.05
- U0.001
- 0.049, 0.051
U0.05
-unlikely that results are due to chance => reject H0, significant result
U0.001
-strong evidence for a significant result
- 049 or 0.051
- ‘borderline’ significant values could suggest to reject H0
What is the difference between a
- type 1 error (a error)
- type 2 error (b error)
T1 a
- occurs when a statistical test rejects a true H0
- false positive
- significance level is set at 5%, so every time we carry out a test, we have a 5% chance of a false positive
T2 b
- test fails to reject a false H0
- false negative
- 1-power
- power set at 80% => b=0.2
Can decrease the T2 error by increasing the sample size of the study => increases power to detect an effect, if a true effect exists
What is the problem with multiple tests within the same dataset
-what are some examples of multiple testing
Each time you perform a test at the conventional significance threshold
a = 0.05, you accept a 5% chance of 1 false positive
More tests performed in same dataset => higher the number of false positives you have to accept
This applies when you have
- multiple outcomes you are measuring
- a single outcome in multiple subgroups
- multiple exposures
How would you avoid multiple testing
Limit the number of primary outcomes when designing the investigation
If this isn’t possible
Present findings as exploratory findings
-this requires further confirmation because the sample size was calculated on the primary outcome only => high risk of false positives due to multiple testing
Present multiple outcomes into 1 composite outcome
At the statistical analysis stage -Bonferroni correction
-this method corrects the significance level a for n (the no of tests performed according to the formula a/n
How well does the Bonferroni correction perform with
- independent hypotheses
- correlated hypotheses
Independent (same hypothesis tested in several different groups of subjects from the same sample)
-performance is good, i.e. we have a good correction
Correlated (same hypothesis tested on related outcomes or exposures
- performance is poor, i.e. we have an overconservative correction
- apply a modified Bonferroni that accounts for the correlation between tests
How would you avoid getting false negatives
Results from low power (small sample sizes)
Wrongly interpreted as no effect
Can be avoided if a priori sample size calculations are performed
Must also account for
- possible drop outs
- the need to correct for multiple testing if necessary
- the minimal detectable difference has to reflect clinically meaningful effects
What is the problem with studies with a sample size that is
- too large
- too small
Too large => likely to produce false positives
Too small => likely to produce false negatives
-aim to design the investigation to avoid this where possible
How would you calculate the sample size of each group
N in each group =
f(a, b) x [2(SD^2)/(m2-m1)^2]
a = possibility of wrongly rejecting H0
b = power
m2, m1 = means of different groups
Describe the relationship between the
- minimal detectable difference and the sample size
- minimal detectable difference and power
If we half the size of the minimal detectable difference => quadruple the sample size
If we increase the minimal detectable difference, we increase the power
How would you do a power calculation
Zpower =
[m2-m1/SD x √(n/2) - Za/2
What is the difference between
- unpaired
- paired variables
Unpaired (independent)
- variable of interest has been measured once in each subject
- the groups we want to compare are independent as they include separate subjects
Paired (dependent)
- variable of interest has been measured more than once in the same subjects but at 2 different time points
- we aim to compare the values of the variable within the same subject
What is the
- H0 for x2 test
- formula
- interpretation
The observed frequencies is the same as the expected frequencies, if there is no association between the 2 variables
X2 = sum of all cells [(O-E)^2/E]
X2 and degrees of freedom needed to find the p value
When would you use Fisher’s exact test
-what is the formula for Fishers exact test
If you have a
-small sample size or
-20%+ of the cells have expected frequencies under 5
then X2 is not used
If you have a 2x2 table, only 1 cell needs to have an expected frequency under 5, don’t use X2
Interpretation of Fisher’s test is the same
(a+b)!(c+d)!(a+c)!(b+d)!/n!a!b!c!d!
What is McNemar’s test formula
-what is it testing
X2 = (|b-c| -1)^2/b+c
Looks at those who move from 1 category to another
What is the Mann-Whitney U test for
- how would you do this
- what is the formula
Tests the difference in the values of an ordinal variable between 2 independent samples
- Create a single list with all values ranking from lowest to highest, distinguishing between both groups
- Add ranks for the 2 groups separately
- if 2+ observations have the same value, they are given a rank equal to the midpoint of the unadjusted rankings - Sum up the ranks of each group separately
U1 = n1n2 + [ n1(n1 + 1)/2 ] - R1 U2= n1n2 + [ n1(n1 + 1)/2 ] - R2
U = test statistic
-is the smaller of U1 and U2
If U is lower than the significance level => statistically significant
What is the sign test used for
-what would the H0 be
Test the difference in the values of pairs of observations for an ordinal variable
-can only say whether the difference is positive or negative
Overall no change between time point 1 and 2 (equal numbers of positive and negative differences)
Compare the number or positive or negative differences to the values in the significance tables
What assumptions are made in the unpaired Student’s t test
- how would you check the assumptions
- formula
- H0 and H1
- what would you do if variances are not similar
Data is
- independent
- normally distributed
- variance of data on compared 2 groups is the same (calculate SD^2 to see if they are the same/similar)
t = mean1 - mean2 / SD √(1/n1) + (1/n2)
t value compared to value in stats tables to assess the probability of getting that result
H0
-there is no difference between the means of both groups
H1
-there is a difference between the means of both groups
Unequal variance unpaired t test uses a different formula
H0
-there is no difference between the mean of 1 group to the individuals in the other group
H1
-there is a difference between the means of 1 group to the individuals in the other group
What are the assumptions that must be met to use the paired Student’s paired t test
-formula
Data is not independent
Difference between paired groups are normally distributed (this is not the same as the distribution of each group is normal)
Does not assume that variance within each paired group is the same
t = mean difference/SE of mean difference
Value compared to value from stats tables to asses for significance
Mean difference is different from the difference between 2 means (x1-x2)
When would you use Wilcoxon signed-rank test
-how would you do this
Test the difference in the values of pairs of observations (dependent samples) for quantitative variables, whenever the assumptions of the parametric test, Student’s paired t test are not met
- Calculate the difference between each paired value
- Rank by size of difference ignoring the sign
- if differences are identical, each is given the mean of the ranks they would have if distinct
- if there are 0 values, they are omitted - Add up the positive differences and negative differences separately
- Test statistic is the smaller of the 2 values, compare this to the value from the stats table
What is Pearson’s Correlation Coefficient
- what does each value mean
- formula
- what assumptions have been made
Rho quantifies the direction and strength of the linear association between 2 continuous variables
-the correlation coefficient
-1 = perfect negative correlation 0 = no association Between 0 and 0.4 = weak Between 0.4 and 0.7 = moderate Between 0.7 and 1 = strong \+1 = perfect positive correlation
∑(x1-mean of x)(y1-mean of y)/√(´∑(x1-mean of x)^2 (´∑(y1-mean of y)^2)
Can calculate a p value
Assumes a
-normal distribution of both variables
-linearity between both variables
If these assumptions are not met => use Spearman Rank or Kendall’s Tau correlation
When would you use Spearman’s Rank
-formula
If the assumptions of the Pearson’s correlation coefficient are not met but you still want to investigate the relationship between 2 variables
Rho is interpreted in the same way as in Pearson
Formula calculated by statistical software
Assumptions and errors with correlation
Strong correlations cannot tell you timeline of which factor affects which
- could be another factor that moderates the effect
- will need simple or multiple regression to further investigate the relationship between X and Y
Cannot use correlation to assess the agreement between 2 different methods and decide they agree because the correlation is high
What does regression tell you
-how does this differ from correlation
Assess the relationship between an outcome and one or more risk factors
-how does the value of an outcome change with a change in the risk factor
Correlation
-what is the relationship between X and Y
Regression
- what is the increase in X per unit increase in Y
- gradient of the regression line
When would you use
- linear regression
- logistic regression
Linear
-used for continuous outcomes
Most common method - least squares regression
-chooses the straight line that minimises the sum of squares of the distance between the line and data points
Y = a + bX + e Y = outcome a = Y intercept b = gradient X = risk factor e = difference between the observed and predicted value of Y
Logistic
-used for binary outcomes
What does linear regression assume
Residuals are normally distributed
The observed values of Y have the same spread around the regression line as X changes
Residuals are independent of each other
What are the 3 links between correlation and regression
- Gradient = Pearson’s correlation coeff x (SDy/SDx)
- Pearson’s correlation coeff^2 explains the amount of variation in Y that is explained by X
- p value of correlation coeff = p value of gradient
How would you calculate odds
-what is the difference between probability and odds
p/1-p
p=probability
Probability
-number of successes/total number of trials
Equal probability = 1 success every 2 trials
Can range from 0 to 1
Odds -number of sueccesses/number of failures Equal odds = 1 success and 1 fail Odds can range from - to + infinity -greater than 1 = success more likely than failure
How would you use odds to calculate the logit
-why do we need logit
log of odds
logit = loge(p/1-p)
logit = a + bX + e
-can substitute Y in the linear regression equation
How would you calculate the odds ratio
OR = ad/bc
or can be calculated from the odds
Odds of disease with exposure/odds of disease without exposure
or can be calculated from logistic regression
e^gradient
What are the advantages of using logistic regression
Gives us a p value for the odds ratio
Gives us a confidence interval for the odds ratio
Can be used to investigate or control for several different factors simultaneously
What are the assumptions of logistic regression
Dependent variable must be binary
-must be coded 0 and 1
Each observation is independent
Needs a large sample (minimum of 20 is good)
How do we interpret the odds ratio with the confidence interval
Does not include 1 = evidence of a relationship between the risk factor and outcome
-p value will be less than 0.05
What is a confounder
A factor that affects a relationship between X and Y
Can be
- a risk factor for the effect
- associated with the study base
- not an intermediate factor between the exposure and effect
How can the confounder affect the results
Systematic error
Positive - overestimate true association, bias away from H0
Negative - underestimate true association, bias toward H0
How do we identify a confounder
If after we add a variable into a statistical model, the crude estimated measure of association between an exposure and outcome varies by 10%+, confounding is found
Slightly different methods of assessing the magnitude of confounding
Crude starting value = RRcrude-RRadjusted / RRcrude
Adjusted starting value = RRcrude-RRadjusted / RRcrude
How to design out confounding
Randomisation
- random allocation of subjects to exposure
- requires large sample size
- only applicable in experimental studies
Restriction
- inclusion of subjects belonging to 1 stratum only of the confounders
- may result in lower statistical power, limits external validity
Matching
- selection of controls so distribution of potential confounders is similar to cases
- expensive, time consuming
How to control for confounding at data analysis
Stratification (Mantel Haenszel method)
- strength of association between exposure and outcome measured in each stratum of the confounder
- weighed average calculated to account for difference in distribution of confounder
- but this reduces power as there will be fewer participants per strata
Multivariable modelling
-statistical modelling that controls multiple confounders at once
What are the possible characteristics we can conclude about a confounder
Strength of effects between confounder and exposure
Direction of effect between confounder and exposure
-positive or negative
May appear to be a risk factor or a protective factor
Why is it not possible to completely eliminate confounding
Limitation of ability to control for all potential confounders
-some are unknown or unmeasurable
Random error in measuring confounders reduces our power to control them
What are the assumptions for multiple regressions model
-formula
Used to control for confounders during data analysis
Same as simple regression
Addition of no strong correlation between predictors
Y = a + b1(factor1) + b2(factor2) + b3(factor3) + e
b’s are adjusted for the effect of each factor respectively
What is the formula for multiple logistical regression
Used to control for confounders during data analysis
logit = a + b1(factor1) + b2(factor2) + b3(factor3) + e
