Critical appraisal Flashcards
1
Q
What is meany by the Number Needed to Treat (NNT)
A
The NNT measures the number of patients who need to receive a treatment to prevent one additional bad outcome
2
Q
equation for NNT
A
In studies it is calculated by 1 / (EER - CER)
i.e. NNT = 1 / ARR
3
Q
what is meant by relative risk
A
RR = Probability of an event when exposed/Probability of event in control group
Also called risk ratio.
measure of proportionate increase in exposed group
RR = EER / CER (= (a / a+c) / (b/b+d) )
4
Q
what is meant by absolute risk?
A
Absolute risk is likelihood of developing a disease over a time period
5
Q
what is meant by hazard ratio
A
A hazard ratio considers your absolute risk to be 1.
If something you do / take doesn’t change your risk, then the hazard ratio is 1.
If something you do lowers your risk by 30% compared to not doing it then that action makes your hazard ratio 0.70 = the risk is 70% of what it was (30% lower).
If something you do triples your risk, then your hazard ratio is 3.0 (risk is 3 times greater than it was before).
6
Q
Experimental event rate calculation
A
EER = a/ (a+c)
a = exposed and diseased c = exposed and not diseased
A measure of how often a particular event occurs within the experimental group of a trial.
7
Q
what is the Absolute risk reduction
A
ARR (RD) = CER - EER
i.e. (a/a+c) / (b/b+d)
absolute risk reduction is the same as risk difference. Comes from subtracting the risk in one group from the risk in the other group.
High risk patients show a bigger risk difference than low risk patients. Patients who are unlikely to experience the outcome will need more patients to be treated for one outcome to be avoided, and the risk difference is smaller.
8
Q
what does it mean if the relative risk is 1
A
no risk difference
no causation
9
Q
what does it mean if the relative risk is > 1
A
Increased risk amongst those exposed
i.e. it is a risk factor
10
Q
what does it mean if the relative risk is < 1
A
decreased risk amongst those exposed
i.e. it is a protective factor
11
Q
what is relative risk reduction
A
Relative risk reduction is the proportional reduction in risk between active and control participants in a trial.
If the relative risk reduction is 10%, the risk in the active treatment group will be 90% of that in the control group.
12
Q
what makes the NNT larger?
A
Less effective treatments
i.e. prevent fewer events, so more patients need to be treated to prevent a single event.
13
Q
what makes NNT smaller?
A
The NNT gets smaller for longer durations of treatment.
Or for more effective treatments
14
Q
Which will appear more impressive of the relative risk or the absolute risk?
A
The relative risk will always be a larger percentage figure than absolute risk, so relative risk seems more impressive.
15
Q
What impact does the framing of research results have
A
The framing of results is the way that results are presented and this can influences doctors’ attitudes about a treatment.
16
Q
what is EER
A
EER = experimental event rate (incidence in group receiving intervention) EER = a/ (a+b)
17
Q
what is CER
A
CER = control event rate (incidence measured outcome in control) CER = c / c+d
18
Q
What is meant by odds ratio
A
Odds Ratio is a method of comparing how likely events are between 2 groups.
odds of event occurring in one group divided by the odds of event occurring in the other group.
OR = a x d / b x c
19
Q
calculation for odds of a positive outcome in exposed patients
A
odds f a positive outcome in exposed patients = a/b
e.g. odds of rolling a 1 on a dice = 1/5 = 0.2
20
Q
what is a Student’s t-Test
A
Students T-test compares two means (averages) and tells you if they are different from each other.
And how significant the differences are.
21
Q
what is Chi Squared test
A
Chi Squared test is a
Nonparametric test used to compare numerical or categorical data sets.
22
Q
what is categorical variable
A
ategorical variables represent types of data which may be divided into groups.
Examples of categorical variables are race, sex, age group, and educational level.
23
Q
what is meant by Nonparametric data
A
Does not fit a normal distribution.
Nonparametric statistics uses data that is often ordinal, meaning it does not rely on numbers, but rather on a ranking or order of sorts.
e.g. a survey conveying consumer preferences ranging from like to dislike would be considered ordinal data.
24
Q
what is a numerical / quantitative (Non-categorical) variable
A
Numerical (Non-categorical) variables are usually referred to as range variables and are expressed as numbers within the set of the Reals, usually varying from negative infinity to positive infinity
25
what statistical tests can be done with data that is normally distributed
Students T test
| ANOVA (analysis of variance)
26
what statistical tests can be done with data that is not normally distributed
nonparametric tests - which dont assume anything about the population parameters.
chi-square,
Fisher’s exact test
Mann-Whitney test.
27
what is a regression analysis used for
A regression equation is used in stats to find out what relationship, if any, exists between sets of data
Regression analysis is also used to understand which among the independent variables are related to the dependent variable.
In restricted circumstances a regression analysis can be used to infer causal relationships between the independent and dependent variables.
28
what does simple linear regression result in
A linear relationship between one dependent variable and one independent variables = a straight line of best fit
Like an average of where all the points line up.
In linear regression, the regression line is a perfectly straight line
29
what does polynomial regression result in
A polynomial regression results in a curved line of best fit .
polynomial regression is considered to be a special case of multiple linear regression
30
what does multiple linear regression result in
Also known as multiple regression
A statistical technique to model the linear relationship between the 2 pr exploratory (independent) variables and the response (dependent) variable.
linear relationship between one dependent variable 2 or more independent variables.
31
What are Rank Sum tests?
Wilcoxan and Mann Whitney U tests are both rank sum tests.
These are Nonparametric tests
Does not require the assumption of normal distributions.
32
What are Bradford Hills criteria and what are they for?
The criteria for causation
1. Temporality - time sequence
2. plausability
3. consistency
4. strength of association
5. dose - response gradient
6. reversibility
7. specificity
8. coherence
9. analogy
33
points to look at when appraising a case control study?
- focused aim / issue
- Is the method appropriate for the question
- Recruitment - case definition, cases representative of population, system for case selection, are cases incident or prevalent, time frame of stuy relevant to disease?, power calculation
- contol selection - representative and matched?
- Exposure accurately measured? recall bias, subjective, is the measure validated
- blinding
- confounders
- results
- strenght of association, OR, P-value, CI
34
what is a consort flow diagram
a flow diagram to provide a broad overview of how the trial was conducted
Concise method to report research method and protocol
Shows progress through phases of an RCT of each trial arm
35
Advantage of case control studies
```
cheap
Quicker than cohort
requires smaller sample size
not prone to loss to follow up
study rare outcomes
```
36
Define positive predictive value
how likely someone who tests positve actually has the disease
= true positive
PPV = A/A+B
(where A is screen +ve has disease and B is screen positive no disease)
37
Define negative predictive value
How likely someone who tests negative actually does not have the diease
= true negative
PPV = D/C+D
(where D is screen -ve no disease and C is screen -ve has disease)
38
what is the attributable risk
Same as the risk difference.
difference between incidence rates in the exposed and non exposed groups
39
calculation for control event rate (CER)
CER = b / b + d
```
b = not exposed and diseased
c = not exposed and not diseased
```
40
calculation for estimated population exposure
a + c / a + b + c + d
41
calculation for (population attributable risk percent) PAR%
PAR% is calculated by
dividing the population attributable risk (PAR) by the incidence in the total population
then multiplying the product by 100 to obtain a percentage
42
How is relative risk different to attributable risk?
Relative risk helps determine whether and how strongly a precursor is associated with a particular outcome.
Attributable risk helps determine how much of an
outcome may be attributable to a particular risk factor (i.e. an estimate of the excess risk) in a
population exposed to that factor.
43
what is a population attributable risk?
The proportion of all cases of an
| outcome in the total population that could be attributed to the exposure to the risk factor.
44
what factors help distinguish cause from association?
```
strength of association (relative risk)
time sequence - cause MUST precede outcome
distribution - geographical
gradient - dose and duration correlation with outcome
consistency
specificity
biologically plausable
experimentally reproducable
precentative trials
```
45
what is the absolute risk?
Number of events (or disease) / total number in the group or population
46
uses of descriptive studies
guidance on association
demonstrate a broad geographical difference
determine relative frequency of dieases or factors
service planning
evaluating service effectiveness
47
types of sampling methods for observational studies
simple random sampling - e.g. random number generator
systematic sampling - seclect person at regular intervals
stratified sample - propabbility of patient selection varies by a pre-determined characteristic to ensure a subgroup is represented
cluster sample - groups are the sampling units, randomly selected
continuous sampling
convenience sampling
48
sources of bias in sampling
deviation from sampling methodology
omission of people who are hard to identify as belonging to the population
large numbers declined to participate
self selection or volunteer patients
excessive exclusion criteria
Berkson bias - selection taken from a sub-population who do not reflect the whole population (e.g hospital patients)
49
What does CONSORT stand for?
Consolidated Standards Of Reporting Trials
50
what is CONSORT used for
Consolidated Standards Of Reporting Trials
A statement introducing a set of recommendations to improve RCT reporting
Protocol to identify problems from conducting RCTs and assist with reporting them fully and clearly
51
what are the 4 stages of an RCT listed in a CONSORT flow diagram?
1. Enrollment
2. Intervention allocation
3. Follow - up
4. Analysis of data / results
52
The first part of a CONSORT flow diagram is 'enrollment'
| what 3 things should this include?
Enrollment - include
1 - Number assessed for eligibility
2 - Number excluded and why - not meeting inclusion criteria / decline / other reasons
3 - Number randomised (study population)
53
The second part of a CONSORT flow diagram is 'allocation'
| what should this include?
Allocation includes
- number of patients allocated to intervention
- number of patients in each arm who receive the intervention
- number in each arm who do not receive the intervention
54
The third part of a CONSORT flow diagram is 'follow-up'
| what should this include?
Follow-up should include - for each arm
- number lost to follow up
- reasons for loss to follow up
- number discontinuing the intervention
- reasons for discontinuing
55
The fourth part of a CONSORT flow diagram is 'analysis'
| what should this include?
Analysis includes - for each arm
- number analysed
- number excluded from analysis
- reasons for exclusion from analysis
56
Disadvantages of a retrospective case control study
```
Retrospective
recall bias + reduced reliability
incomplete records
previous records not standardised
Difficult to prove causality
Difficult to select appropriate controls
Cannot calculate incidence within a population
```
57
Why is an intention to treat analysis usually done?
What is the benefit of this?
subjects are analysed according to the treatment allocation irrespective of whether is was recieved
This is because treatment allocation was random.
Any changes to treatment were not random.
ITT avoids breaking randomisatin
Therefore not doing an ITT analysis introduces a potential for confounding.
ITT analysis demonstrates the potential effect of the treatment protocol rather than the specific treatment.
58
What is a 'rate' in relation to studies
The relative frequency of an event per unit of time
The calendar time period must be the same for the numerator and denominator
often expressed as per 1000 per year
59
What is PICO
Population / patients
Intervention
Control / comparator
Outcome
60
In PICO what should be considered in relation to 'P'
Population / patients
- age
- gender
- ethnicity
- diagnosis
61
In PICO what should be considered in relation to 'I'
Intervention
| - the treatment or exposure
62
In PICO what should be considered in relation to 'C'
Control / comparison
- control / non exposed
- placebo
- current best treatment
63
In PICO what should be considered in relation to 'O'
Outcome - must be measurable
- pain score
- survival time
- cure / remission
- shorter hospital stay
64
Points to look at when appraising an RCT
```
Aims and primary outcome - clarity
Recruitment and selection
Randomisation
Allocation concealment
Controls and matching
Confounders
Exclusion criteria
Blinding
Drop out rates - why
sample size - power calculation
results - p-values, CI
ITT analysis or per-protocol
```
65
what is the risk of an under powered study
A small sample size leads to an under-powered study.
This increases the risk of making a type 1 error
= where it is believed that a differnce exists where no such difference does exist
66
What is a type 1 error
where researchers believe an effect is demonstrated when really there is no effect
I.e to incorrectly reject the null hypothesis
67
Methods of correcting for multiple statistical analyses
Bonferroni - P-value / number of tests
Benjamini-Hochberg procedure - involves ranked p-values from smallest to largest and allocating the smallest rank to one, rank 2 to the next and so on. Then compare each to the B-H critical value
68
Sources of error
1. Subject Variation = differences in the observed factor
- e.g. variation in BP on different occasions
- impact of motivation, recall etc
2. Observer variation
- intra-observer variation
- inter-observer variation
- bias due to awareness of the hypothesis
- inconsistent procedure or question wording
- lack of skill / training / interest
3. Technical Limitations
- Test doesnt measure what it is intended to
- method intrinsically unreliable
- defective instruments
- incorrect calibration
- test repetition and replication
69
Advantages of prospective cohort studies
Prospective = better able to standardise methods and reduce error
Can calculate incidence in exposed and non-exposed groups
Can distinguish cause from association
Can study several outcomes from one hazard
70
disadvantages of prospective cohort studies
Long duration
Large number of subjects
Expensive
May have higher drop out rate / loss to follow up
cannot be certain of true causation - this requires experimentation
71
Why is it usually preferable for a study to recruit incident rather than prevalent cases?
Incident cases are newly diagnosed and have less risk of confounders
Prevalent cases are patients with already established disease - therefore they represent the survivors of an earlier cohort of incident cases.
Prevelent cases are less likely to represent everyone who is diagnosed as it doesn't include those who get better or die.
72
management of confounding variables in a case-control study
matching of controls
exclusion criteria
subset analysis based on confounding variables
multivariate analysis technique e.g. logistic regression
73
Calculation of Rate %
(number of cases / population at risk) x 100
74
Hierarchy of evidence
```
Meta analysis / Systematic review
RCT
Non randomised CT
Cohort study
Case control study
Cross sectional survey
Case series
Case report
Editorials, expert oppinions
Animal research
```
75
Define sensitivity
the ability of the test to correctly identify those patients with the disease.
= patient does have the disease and is positive on the test
A/ a+c
76
define specificity
the ability of the test to correctly identify those patients without the disease.
= patient does not have the disease and is negative on the test
77
What does the positive predictive value depend upon?
PPV and NPV are dependent on the population being tested
| Are influenced by the prevalence of the disease
78
Define likelihood ratio
How much more likely is it that a patient who tests positive has the disease compared with one who tests negative.
likelihood that a given test result would be expected in a patient with the target disorder compared to the likelihood that that same result would be expected in a patient without the target disorder.
79
5 steps of EBM
1) form a question re a clinical problem
2) evidence
3) critical appraisal
4) application
5) implementation and monitoring
80
Items to consider if appraising a website
```
Is there a named author?
Author qualifications?
Funding / supporting of site listed?
Any conflicts of interest recorded?
Is the site hosted by a reputable organisation - e.g. .org / .gov / .edu
Any peer review process
Is the coverage full and unbiased
does the spelling, grammar and information appear accurate?
```
81
What does the term 'MeSH search' mean?
Medical Subject Headings search - uses pre-defined keywords
82
Types of observational descriptive studies
Survey / questionaire
Qualitative survey
Case series
Case report
83
Types of observational analytical studies
Case control study
| Cohort study
84
Types of experimental studies
```
Randomised control trial
Controlled trial
Uncontrolled trial
Cross over trial
N of 1 trial
```
85
What is an ecological study
A study on a population or community
86
What is a pragmatic trial?
Takes place in a real life environment therefore may be more reflective of everyday practce
87
what is an economic analysis
Assess cost and/or utilities of intervention
| Helps to prioritize services
88
What is a systematic review
A group of studies which are grouped and appraised together to answer a hypothesis
89
What is a meta-analysis
A statistical procedure for combining the numerical data of separate `studies to answer a clinical question
90
What is internal validity
To what extent does the study measure what it set out to measure.
What is the evidence that the intervention caused the outcome
Did the study limit and control confounding variables
Robust research design
Sufficient power
Correct statistical analysis
91
Factors affecting internal validity
```
Attrition / mortality (drop outs)
maturation - respondants grow out of the condition / grow experinenced
instrumentation
regression towards the mean
selection
contamination
history
per-protocol analysis
```
92
What is external validity
The extent to which results from the study can be generalised to the wider population
Can the results be applied to real patients in real life settings
Can the results be generalised to other locations and times
93
Factors affecting external validity
Sample population representative of wider population?
Sampling method
Does the test / measurement affect behavior or outcome
does the test environment mirror real life
is the test stimuli similar to real life
do the elicited behaviours represent real life behaviours
94
Define efficacy vs effectiveness
Efficacy = the impact of interventions under optimal trial conditions
Effectiveness = the impact of the interventions under clinical or real life circumstances
95
Considerations regarding the journal when appraising evidence
Is it peer reviewed - independently, non-editorial staff
Is peer review double-blind?
Impact factor - based on number of citations
96
Disadvantages of the peer review system?
delay in publication
author may be known to the reviewer - therefore reduces objectivity
revolutionary / unpopular opinions may face opposition within peer review
doesnt guarantee errors will not appear
97
How is a journal impact factor calculated?
Ranking of journals by measuring the frequency of the average article cited per year
Done by reuters
Comparison of journal performance
98
What is the Indelfinger rule?
Neither original research
or any of its pictures or tables
may be published in more than one outlet
99
Types of conflict of interest
Factors which may influence professional judgements
Financial - grants / honoraria
Professional - membership to an organisation
Personal
100
criteria of authorship
Any of the following
- Substantial contribution to conception
- or design of the work
- or acquisition / analysis / interpretation of data for the work
101
Risk of multiple subgroup analysis
'data dredging'
Increased risk of Type 1 Error
more analyses = more likely that a significant result result will be found by chance
- unless the p-value is corrected
102
Risk associated with adding subgroup analyses after the data is collected
Introduces bias
increases risk of making a type 1 error
research method and power may be sub-optimal to answer additional questions
103
Difference between bias and confounding
Bias leads to wrong results - the data is incorrect
Confounding leads to the wrong conclusion - but the results are correct
104
What causes bias
Mistakes in the research techniques
The results generated are therefore wrong
Bias is not a real-life association or relationship but is introduced by the research process
105
What causes confounding?
A real-life relationship that exists between exposure and outcome independent of the research process.
NOT a mistake
Studies have the correct result but the wrong conclusion may be reached if confounders are not recognized
106
2 common types of bias
Selection bias
| Observation bias
107
What is selection bias
Problem with recruitment and allocation
Sample not representative of target population
includes - sampling bias and response bias
108
What is observation bias
Problem with the way data is gathered
| Results influenced by expectations of research team or subjects
109
Types of selection bias
```
Sampling bias
Response bias
Berkson / admission rate bias
Diagnostic purity bias
Neyman bias / incidence-prevalence bias
Membership bias
Historical control bias
```
110
What is a Berkson Bias
Admission rate bias
e.g. sample taken from inpatient setting but the hospital cases do not represent the severity of the condition in the general population
111
What is a diagnostic purity bias?
Excessive exclusion criteria
Comorbidity is excluded from the sample to such an extent that the sample doesnt reflect the complexities of the general population
112
What is a Neyman bias?
Incidence-prevalance bias
Occurs if prevalence of a condition doesnt reflect its incidence
Often affects rapidly progresssive conditions
Gap between onset of the condition and selection of sample means some individuals with the condition are no longer available due to death
Therefore sample = milder form - but generalised to whole population
113
What is a membership bias
When membership to a group is used to recruit subjects - members may not be representative of the population due to motivation etc
114
What is historical control bias
Subjects and controls are selected across time
Changes in definition, exposures, diseases and treatment can mean that the subjects and controls cannot validly be compared
115
What is a response bias?
type of selection bias
Individuals volunteer for the study but differ in some way from the general population
e.g volunteers are motivated
can arise from advertising study e.g. radio / papers
116
Types of observation bias
Interviewer / ascertainment bias
recall bias
response bias
hawthorne effect
117
What is Interviewer / ascertainment bias
Researcher is not blinded to the subjects status in the trial which affects their approach to recording results
118
What is recall bias
Subjects selevtively recall details from the past
| Esp in case control studies or surveys
119
What is response bias
Can occur in any study where the patient is asked questions
| Patients may answer in the way they think the researcher wants to hear - rather than their true feelings ./ beliefs
120
what is the Hawthorne effect
Subjects alter their behaviour when they are aware they they are being observed in a study
121
What is performance bias?
Systematic differences exist between groups in the care provided
or in exposure to factors other than the intervention of interest
Reduced by
- blinding
- randomization
- standardization of care protocols
122
What is attrition bias
When the numbers of individuals dropping out of the studies differ significantly between the study arms
Incomplete outcome data
Those at the end may not be representative of those who started and the target population
123
What is a qualitative survey?
Gathering opinions of a group of people.
Emphasis on subjective meaning and experience
Used to study complex issues
124
What is a case report
The experience of a single person described
Usually anecdotal
Usually cannot be replicated
Can be used to generate a hypothesis
125
What is a case series?
The experiences of a group of people is described
| Useful for studying rare diseases
126
Can you critically appraise a case study or case series?
Not really
But consider
- Did the study address a clearly focused question / issue?
- Is the researcher’s perspective described / taken into account?
- Is the patients perspective described / taken into account?
- Are the conclusions drawn justified by the case?
- Are the findings of the study relevant to other settings?
127
What is an observational analytical study
Reports similarities and differences observed between 2 or more groups
- case control study
- cohort study
128
What is a case control study
Type of observational analytical study
Retrospective
People who have the outcome variable are compared to people who do not - to determine risk factors they have been exposed to in the past
129
Advantages of a case control study
Cheap
Easier than other studies
few subjects required
suitable for studying rare outcomes / those with a long time lag between exposure and outcome
130
Disadvantages of a case control study
Rely on recall or health records = source of bias
Unsuitable for rare exposures as this would require a v large sample
can be difficult to select a control group - need to be similar to the cases group
131
What is a cohort study
Type of observational analytical study
Prospective
Group of people followed to see how development of n outcomes differs between those with an exposure and those without
132
Advantages of a cohort study
Useful for rare exposures
can assess multiple outcomes
can assess temporal relationships between risk factors and outcomes
can directly estimate outcome and incidence rates
133
Disadvantages of a cohort study
May take a long time from exposure to outcome
Expensive
Problems with dropu outs / loss to follow up
unsuitable for rare outcomes as would need a v. large sample
134
How is a 'retrospective cohort study' conducted?
Using a pre-existing cohort
Use a historical group of patients who had an exposure and contact them now to see if they have developed the outcome of interest - avoids need to wait
