AFP Critical Appraisal & Stats Flashcards

1
Q

How will you summarise what the paper is about?

A
Population = People with AF
Intervention = Apixaban 5mg BD
Control = Warfarin INR 2-3
Outcomes = Stroke or systemic embolism
Key findings
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What should you think about before summarising the paper?

A

QR

What is the research q and what is its relevance to clinical practise

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What acronym do you use to assess internal validity?

A
Recruitment
Allocation
Maintenance
Baseline
Outcome (and was it blinded)
Stats
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What is internal validity?

A

The extent to which the observed results represent the truth in the population we are studying and, thus, are not due to methodological errors

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

How can we reduce selection bias?

A

Use consecutive recruitment rather than non-consecutive

Consider the recruitment location (e.g. GP vs hospital)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What types of recruitment are there?

A

Consecutive vs non-consecutive

Single centre vs multicentre

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Types of allocation?

A

Randomised vs non randomised
Allocation blinded vs open label
Letters/Packets vs automated voice recognition system (electronic)
Block randomisation vs whole group randomisation
Cluster randomisation vs whole group randomisation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What is block randomisation?

A

Wait for X number of study participants and then say 50% go to each arm
Throughout the whole study there are equal numbers in both arms
Helps to mitigate temporal trends

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What is cluster randomisation? Disadvantages?

A

Used in multicentre studies
Used when you can’t deliver different interventions in one place (e.g. poster in a GP practice)
Needs higher numbers of people
More difficult to achieve balance across groups

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Benefit of randomisation?

A

Reduces risk of confounders

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Why does blinding help?

A

Reduces risk of placebo effect

Reduces bias in data analysis

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Benefit of Electronic voice recognition system?

A

Minimizes risk of tampering

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What is study maintenance and what is usually aimed for?

A

Maintenance is the drop out rate from a study

Usually aim for <20% but ideally <10%

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What bias is created if there is a poor maintenance?

A

Attrition bias - causes study to be under powered

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Types of blinded outcome?

A

Single: patient only
Double: patient and physicians interacting
Triple: patient, physicians, outcome extractors

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

How can you blind if the outcome can’t be blinded e.g. surgical?

A

Use PROBE design
Prospective Randomised Open label Blinded endpoint adjudication
Whoever reviews the outcomes doesn’t know which arm of study the patient is in (e.g. radiologist when looking at stroke tx)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

What do you need to look at in baseline?

A

Baseline characteristics of population and are they matched?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Besides blinding what else do you need to consider with outcome?

A

Is the choice of outcome important, adequate and important to patients?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Define power

A

the likelihood of detecting a difference when it exists (avoiding Type 2 error)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What 4 things should we look at when analysing the stats of a paper?

A

Power calculation and according recruitment target
Statistical models used for outcome measure (binary, continuous, time-dependent)
Effect size and significance level
Absolute risk reduction / Relative risk reduction / NNT

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

What 3 things does power calculation depend on?

A
  1. How many events are expected over follow-up time (incidence x time) - increase sample size
  2. Expected improvement (RRR of 50%) - increase effect size
  3. What probability you want to detect the difference when it exists (80%?) - increase precision of measurement
    If a study is negative then think about the power!
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Define p value

A

probability that the association detected has arisen by chance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

How to strengthen the p value?

A

Size of association

Size of cohort

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

What test do you use to calculate difference between 2 groups with categorical variables when not adjusted? E.g. stroke vs no stroke

A

Chi-squared

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
What test do you use to calculate difference between 2 groups with categorical variables when adjusted? E.g AF X Stroke
Binary logistic regression
26
What test do you use for comparing 2 groups with a continuous outcome e.g. Gender x BMI if normally distributed?
T-test
27
What test do you use for comparing 2 groups with a continuous outcome e.g. Gender x BMI if not normally distributed?
Wilcoxon ranked test (dependent)/Mann Whitney U (independent)
28
What test do you use for comparing >2 groups with a continuous outcome e.g. number of children x BMI if adjusted?
ANOVA
29
What test do you use for comparing >2 groups with a continuous outcome e.g. number of children x BMI if not adjusted?
ANCOVA
30
What test do you use to find a correlation between 2 parametric continuous variables?
Pearson's rank
31
What test do you use to find a correlation between 2 non-parametric continuous variables?
Spearman's rank
32
What test to use for time to event data i.e. when time counts (e.g. mortality in cancer tx) to determine a direct association?
Kaplan Meier analysis
33
What test to use for time to event data i.e. when time counts (e.g. mortality in cancer tx) when adjusted for covariates?
Cox proportional hazards model
34
How to assess external validity?
Resources - equipment/cost | Population - can you generalise the demographics
35
What are the last 3 things we consider?
Funding - any conflicts of interest Ethics - clinical equipoise, safety outcomes, data safety monitoring board Conclusion
36
Define 95% Confidence interval
A range, between which the population mean value will lie 95% of the time
37
Define relative risk
Risk of developing disease in the exposed group compared to the risk of developing disease in the unexposed group
38
How do you calculate relative risk?
Those who got disease in exposed group/all exposed divided by those who got disease in non-exposed group/all non-exposed
39
Which studies is relative risk used in?
Prospective cohort
40
Define odds ratio?
Odds of something happening vs the odds of it not happening
41
How do you calculate odds ratio?
Exposed with disease/not exposed with disease divided by exposed without disease/not exposed without disease AKA odds in disease group/odds in control
42
What studies is odds ratio used in?
Retrospective observational study (case-control study)
43
Define hazard ratio
Used to look at survival over time with a Kaplan Meier curve - equivalent to relative risk but risk is not constant with respect to time
44
What is a hazard ratio used in?
Prospective RCT
45
Define incidence rate
Number of new cases over a defined period of time
46
Define prevalence
Number of cases in a given population at any given time
47
Define Absolute risk reduction (ARR) and how to calculate it
Difference in the incidence of disease between two groups | = P(event occurring in group 1) – P(event occurring in group 2)
48
Define NNT and how to calculate it
Number of patients who need to be treated to prevent one event occurring = 1/ARR
49
How to calculate RRR?
= ARR / P(event occurring in control group) | Express as %
50
Define Type I error
Probability of rejecting H0 when in fact H0 is true i.e. probability of concluding there is a significant difference when actually there’s no difference. False positive
51
Define Type II error
Probability of concluding H0 is true when in fact it is FALSE i.e. false negative
52
How do we set a type II error?
β is the probability of making type II error – usually set at 0.8
53
How do we set a type I error?
Reflects p value cut-off i.e. usually = 0.05.
54
What does per protocol analysis mean? What is the benefit? What is the con?
Only participants who complied with study protocol completely are included in analysis. More accurate representation of treatment effect Susceptible to attrition bias and exclusion bias
55
What does intention to treat analysis involve? What is the benefit? What is the con?
All participants who have been randomised are included, regardless if they took the medicine / completed the study More accurate representation of effect in clinical practice Can't determine what the drug does in optimal conditions
56
What are case control studies used for?
Identifying associations between exposure and outcome
57
Advantages of case control studies?
``` Rare diseases Quicker and cheaper to perform Can analyse multiple exposures at once Can calculate OR Good for dynamic populations where follow up is difficult ```
58
Disadvantages of case control studies?
Rely on quality of records Selection bias and recall bias Can't demonstrate temporal association
59
What is a cohort study used for?
Evidence for causation and temporal association
60
Advantages of cohort studies?
``` Assess temporality Assess prognosis Can control data collection and quality Can calculate incidence – allows you to calculate RR, risk difference, NNT Good for common exposures ```
61
Disadvantages of cohort studies?
``` Expensive Take longer Not good for rare diseases Loss to follow-up and potential of attrition bias Can be affected by confounders ```
62
Define bias
Factors which cause systematic over or under-estimate of a particular result
63
What is selection bias?
Study sample does not represent entire population
64
What is volunteer bias?
People who volunteer are different from population as a whole
65
What is channelling bias?
Healthcare professionals may subconsciously select patients who would be good for study
66
What is Interview bias?
If interviewer already knows disease status of participant
67
What is recall bias?
Participants remember things differently
68
What is Hawthorne bias?
Participants alter behaviour because they are aware of monitoring
69
What is a confounder?
A factor which has an independent effect on both the exposure and the outcome
70
How can we minimise confounders?
Stratification of participant groups Regression analysis Randomisation and matching
71
What is external validity?
Generalisability of results
72
Difference between RR and HR
RR looks at if an event has occurred during the study | HR looks at if an event occurred and when it occurred
73
How to describe RR of 1.45?
45% more likely to have outcome X
74
How to describe OR of 1.6?
Odds of exposure to factor X is 1.6x higher
75
How to describe HR of 0.79?
At any particular point, group A were 21% less likely to have outcome X
76
What is a type 1 error?
``` Reject H0 (no difference) when actually H0 is true I.e. False positive ```
77
What is a type 2 error?
False negative
78
Define sensitivity and how to calculate it
Probability of correctly identifying those with disease | TP/TP + FN
79
Define specificity and how to calculate it
Probability of correctly identifying those without disease | TN/TN + FP
80
What is a QALY? What is the accepted cost?
Quality adjusted life year | <20k per QALY – ACCEPT. 20-30 – think about
81
What is a fixed effect meta analysis?
Assumes that studies are homogenous All measuring same treatment effect Calculate weighted average - give more weight to bigger studies
82
What is a random effects meta?
Assumes heterogeneity between studies
83
What methods are used to quantify heterogeneity?
Q – statistic - low p = heterogenetic | I2 – estimates proportion of total variance that is attributable to heterogeneity
84
How is heterogeneity dealt with?
Sub group analysis – separate meta analyses | Meta regression - quantify how treatment effect changes with study characteristic
85
What analysis approaches may be performed in an RCT?
Intention to treat Per protocol As treated
86
What is a sensitivity analysis?
Assesses the impact of different assumptions and methodological choices on the results of the primary analysis Can looks at differences in analysis approach, inclusion criteria, outcome definition, missing data May also be good for missing data – can account for it using imputation
87
What groups are responsible for overseeing a trial?
Trial management group – day to day Data monitoring committee – independent – make recommendations. Perform formal interim analysis – stop trial if efficacy clear or serious adverse events Trial steering committee – uses recommendation of DMC
88
What are the issues with a formal interim analysis?
May stop trial prematurely due to random variation in treatment effect over time
89
What is PPV?
Likelihood of people who test positive actually having disease TP/TP + FP
90
What is NPV?
Likelihood of people who test negative actually not having disease TN/TN + FN
91
What is heterogeneity?
the degree of difference between methodology in the studies and thus the treatment effect being measured
92
What does asymmetry in a funnel plot (meta analysis) suggest and what does it lead to?
``` publication bias (only studies that get good results are published) Leads to overestimation of the treatment effect ```
93
How do you overcome publication bias?
Register trial beforehand | Publish a protocol
94
What is clinical equipoise?
a state of genuine uncertainty on the part of the clinical investigator regarding the comparative therapeutic merits of each arm in a trial
95
What is Ecological fallacy and give an example
Assuming that the results of a cross sectional study relate to an individual. Eg areas with higher salt consumption are more at risk of heart attacks. Does amount of salt consumption actually increase likelihood in an individual?
96
How can you graphically represent a meta analysis?
Forest plot
97
What is the difference between a retrospective cohort study and a case control study?
case control takes patients based on outcome status whereas retrospective cohort doesn’t
98
Disadvantage of retrospective cohort study?
Potential to find reverse causality | Recall and interviewer bias
99
Disadvantage of using 2:1 allocation?
Undermines clinical equipoise and can cause therapeutic mis-estimation Reduces power or requires 12% more participants Affects internal validity
100
Advantages of using 2:1 allocation?
In early phase trials to determine clinical utility If a tx is particularly expensive Need for additional safety info - good for adverse events
101
How to interpret a confidence interval looking at the difference between 2 groups?
If it contains the value 0 then p>0.05
102
How to interpret a confidence interval when comparing groups using a ratio instead of a difference e.g. odds/RR?
If it contains the value 1 then p>0.05
103
Difference between OR and RR?
odds ratio is a ratio of two odds whereas the relative risk is a ratio of two probabilities
104
Where can you get research funding from?
``` Non-commercial: Research councils - HRA, MRC Government NIHR Charities - CRUK ``` Commercial: Pharma companies Industry companies
105
Where do you apply for ethical approval?
Health Research Authority Research Ethics Committee
106
Which studies is odds ratio used in?
case control
107
Which stats test to measure strength of association between 2 variables? (ND and NND)
ND: Pearson correlation NND: Spearman rank
108
Which descriptive stats for ND vs NND?
ND: Mean, SD NND: Median, IQR
109
Which stats test for comparison of one group to hypothetical value? ND, NND and categorical
ND: one sample T-test NND: Wilcoxon Categorical: Chi square/binomial
110
Difference between one sample T-test and independent T-test? What is a paired T-test?
One sample compares a group to a pre-determined value Independent compares a group to another group Paired compares values within a group at different intervals
111
Stats test for comparing 2 unpaired groups if ND, NND or categorical?
ND: Unpaired T-test NND: Mann Whitney U Categorical: Fisher's exact test or Chi-squared if large groups
112
Stats test for comparing 2 paired groups if ND, NND or categorical?
ND: Paired T-test NND: Wilcoxon Categorical: McNemar's
113
Stats test for comparing 3+ unmatched groups if ND, NND or categorical?
ND: one way ANOVA NND: Kruskal Wallis Categorical: Chi squared
114
Stats test for comparing 3+ matched groups if ND, NND or categorical?
ND: Repeated measures ANOVA NND: Friedman test Categorical: Cochrane Q
115
What stats test to use for prediction?
Single variable: Linear/logistic regression Multiple variables: Multiple linear/logistic regressions
116
What outcome does logistic regression provide?
Adjusted odds ratio
117
What to use when the denominator is number of person-years?
Relative risk
118
What is the advantage of using parametric tests over non-parametric?
Provides more statistical power
119
What is the positive likelihood ratio?
The probability that a person with the disease tested positive for the disease divided by the probability that a person without the disease tested positive for the disease AKA True positive / False positive LR+ = sensitivity / (1 - specificity)
120
What is the negative likelihood ratio?
False negatives / True negatives LR- = (1 - sensitivity) / specificity
121
Why is it important to do post hoc tests to investigate a significant ANOVA test?
They apply a correction for multiple testing to avoid a type 1 error
122
What are likelihood ratios used for?
They assess the potential use of a diagnostic test and the likelihood of the patient having the disease A ratio of 0-1 reduces chance of disease Ratio >1 increases probability of having disease