EBM

Question 1

what is critical appraisal (critical evaluation)?

Answer 1

the process of carefully and systemically examining research to judge its validity, results and clinical relevance - these are the three most important aspects to consider for critical appraisal

Question 2

what is validity?

Answer 2

This is essentially how close to the truth – or how ‘trustworthy’ – the study is, or how well it measures what it’s supposed to measure. If the researchers have done everything possible to minimise potential sources of bias, then this improves the validity of the study.

Question 3

what is internal validity? what threatens it?

Answer 3

• the internal validity of a study refers to the integrity of the experimental design
• was the experimental design appropriate?
• internal validity is threatened by biases, i.e. a study that is sufficiently free from bias is said to have internal validity

Question 4

what is external validity?

Answer 4

• the external validity of a study refers to the appropriateness by which its results can be applied to non-study patients or populations

Question 5

what is The Critical Appraisal Checklist?

Answer 5

1. Was the study original?
   Consider whether or not the study adds anything to the literature.
2. Who was the study about?
   Consider inclusion/exclusion criteria, how participants were recruited.
3. Was the study design sensible?
   Consider what the intervention or treatment was, and what it was compared to.
   Also consider what outcome was measured (and how it was measured).
4. Was bias avoided or minimised?
   Consider, for example:
   - whether a control group was used (and if the groups were alike).
   - whether adequate randomisation was achieved? (If randomisation was done).
   - any other sources of bias?
5. Was assessment blind?

6.  Were preliminary statistical questions addressed?
	Consider, for example:
-	the size of the sample.
-	the duration of the follow-up.
-	how complete was the follow-up?

Question 6

what are other critical appraisal techniques?

Answer 6

• CASP (Critial Appraisal Skills Programme)
• CEBM (The Oxford Centre for EBM)
• BestBETs - provides comprehensive critical appraisal checklists

systems to improve quality of published evidence that can also be used for critical appraisal:

• CONSORT
• STROBE

Question 7

what does CASP do?

Answer 7

• provides critical appraisal checklists that are designed to help you think about aspects of appraisal systematically
• different checklists are provided for different types of study (e.g., RCT, cohort, case-control study, systematic review).

Question 8

what does CONSORT do?

Answer 8

• focuses specifically on RCTs, listing all aspects that should be included in a good quality RCT
• this checklist can be used either to critically evaluate an RCT or to design a methodologically sound RCT.

Question 9

what does STROBE do?

Answer 9

• provides guidelines for designing observational studies (e.g., cohort studies)
• can also be used for critically evaluating such study designs.

Question 10

what does CEBM do?

Answer 10

suggests levels of evidence (LOE) according to the study designs and critical appraisal of prevention, diagnosis, prognosis, therapy, and harm studies.

Question 11

what is the hierarchy of evidence in terms of type of study? (top to bottom)

Answer 11

• systematic reviews and meta-analyses
• randomised controlled trials
• cohort studies
• case-control studies
• cross-sectional surveys
• ecological studies
• case series and case reports
• ideas, editorials and opinions

Question 12

what is a widely used system for ranking the quality of evidence? what is another system?

Answer 12

GRADE - Grading of Recommendations Assessment, Development and Evaluation

SIGN - Scottish Intercollegiate Guidelines Network provides a slightly different approach

Question 13

what do GRADE and SIGN systems take into account? what is the purpose of them?

Answer 13

The GRADE and SIGN systems both take into account more dimensions than just the quality of medical evidence.

Despite the differences between systems, the purposes are the same: to guide users of clinical research information about which studies are likely to be most valid. It should be noted, though, that the individual studies still require careful critical appraisal.

They’re about ranking the quality of evidence

Question 14

what is bias?

Answer 14

Anything that influences the results of a study (or their interpretation) other than the experimental intervention

Question 15

what are the different types of bias?

Answer 15

pre-trial:

• selection bias
• definition bias - shouldn’t be any ambiguity
• bias in concepts - lack of clarity of concepts in proposed research
• bias due to concurrent disease

during trial:

• information bias
• instruction bias - unclear or no instructions prepared
• lead-time bias - different stages of disease
• attrition bias - differential non response in various groups
• the Hawthorne effect - change behaviour

post-trial:

• confounding bias
• statistical bias

Question 16

what is selection bias? when occurs? how minimised?

Answer 16

• a bias in selection or assignment of patients for a study that arises from study design rather than by change

occurs:

• when the subject studied are not representative of the target population
• when the study and control groups are chosen so that they differ from each other by one or more factors that may affect the outcome of the study

Selection bias can be minimised by random determination of who will be included in the study (provided they meet the inclusion criteria), and by including a large sample size of participants – this helps to ensure that the outcome of the study is representative of the population. Adequate randomisation of participants to different groups in the study should be used to avoid the selection bias associated with assignment of participants to different groups.

Question 17

what is information bias? what does it lead to? what does it include? how can it be minimised?

Answer 17

• errors in measuring exposure/intervention or outcome (e.g. disease)
• leads to systematic difference between groups of participants in the study in terms of the information that is recorded
• includes interviewer bias, recall bias, response bias

Information bias can be minimised with blinding (where observers / assessors do not know which group participants are in), by using objective assessment measures, and standardising assessment procedures.

Question 18

what is confounding bias?

Answer 18

when the outcome of the study is influenced by factors that are associated with the intervention/exposure

Question 19

what is statistical bias?

Answer 19

when the analysis of the results of the study is conducted in such a way as to lead to misleading conclusions (e.g., due to low statistical power or inappropriate statistical tests)

Question 20

what are binary variables?

Answer 20

• a type of non-numerical or ‘categorical’ variable

- measured as one of two possible outcomes - e.g. dead or alive at end of study

Question 21

what studies can t-test and ANOVA be used for? when is each used?

Answer 21

ones with continuous dependent variable & comparable groups

• unpaired t-test to test difference between the means of 2 groups (upaired because you’ve got two groups of different people)
• ANOVA (Analysis of Variance) used for more than 2 treatment groups

Question 22

when can Chi-square test be used? what used for?

Answer 22

for categorical (e.g. binary) dependent variable (& non-continuous independent variable) & comparable groups

• used to determine if observed frequency counts differ from expected frequency counts
• determines whether there is an association between the outcome and the groups

Question 23

what is correlation and regression?

Answer 23

In a cohort study, and with a continuous dependent/outcome variable (e.g., distance walked in 2 min), you might have 2 or more groups being compared (e.g., low, moderate, high exercise) but they almost certainly won’t be alike/similar because they haven’t been randomised into groups. Or – the cohort might not be divided into groups. For example, if everyone uses a pedometer, you might know everybody’s average steps per day. Then we would have a continuous independent variable. Either way, you should use linear regression.
We’ve seen simple correlation and regression before, where 2 variables are related to each other. Regression allows us to predict (or estimate) the dependent variable if we know the independent variable. For example, if we know someone’s height, we can estimate their lung capacity.
But sometimes we’re dealing with more than 2 variables (e.g., 3 or more). For example, if we know participants’ average steps per day, this might be related to the outcome measure of distance walked in 2 minutes. So, to some extent, we can predict the outcome from pedometer performance. However, other factors could be involved. For example, the age of participants could be related to the outcome measure – there may be a tendency for younger people to be able to walk further in 2 minutes. Also, the average number of steps per day would probably be higher for younger people. In this case, age would be a confounding variable (because age relates both to the outcome and an independent variable).

Question 24

what is linear regression?

Answer 24

• used to assess how one or more predictor variables can be used to predict a continuous outcome variable
• allows us to ‘adjust’ for the effects of confounding variables
• “Do age, gender, and comorbidities predict distance walked in 2 minutes following physiotherapy?”

Question 25

what is logistic regression?

Answer 25

• used to assess the predictive value of one or more variables on a binary outcome variable
• Also allows adjustment for effects of confounders.
• “Do age, gender, and comorbidities predict which stroke patients will show improvement following physio?”

Question 26

what is linear and logistic regression useful for? what is difference?

Answer 26

Linear regression allows us to estimate the effects of confounding variables on the outcome. And because the effect can be estimated, it can be adjusted for. This essentially subtracts the effect of the confounding variable (e.g., age) – it basically removes it from the equation.
Logistic regression does the same thing, but it’s used when the outcome variable is not continuous (such as a binary outcome).

(normally both have continuous independent variable)

Question 27

what is the nocebo effect?

Answer 27

adverse psychological or psychophysiological effects produced by placebos

Question 28

how does the placebo effect work?

Answer 28

Expectancies:

• Via outcome expectancies (beliefs that treatments will have positive effects on health status)
• Via patient-related self-efficacy expectancies (beliefs that one can carry out the actions necessary for successful management of a disease)
• Conditioning (e.g., past associations with the context in which health care is delivered)
• Anxiety reduction, social support etc. may have positive follow-on consequences
• Psychophysiological mechanisms (psychological states affecting physiology – e.g., changes in brain chemistry)