Understanding Research Evidence

Question 1

What type of evidence do we look for?

Answer 1

Peer-reviewed publications and/or systematic reviews.

Study design (PICO) must be relevant for what is being looked for. Authors sometimes incorrectly define their designs

Consider possible study limitations and bias

Statistical power

Question 2

What is a randomised controlled trial?

Answer 2

An experimental study (not observational) where peoeple are randomly allocated.

Question 3

What are the types of observational studies?

Answer 3

If there is no comparison group it is a descriptive study.

If there is a comparison group:

If exposure -> outcome it is a cohort study (eg what has eating apples done to people)

If outcome ->exposure it is a case control study (what is causing these strokes?)

If exposure and outcome are at the same time it is a cross-sectional study

Question 4

What are the benefits of conducting a randomised controlled trial?

Answer 4

The gold standard in clinical research

Designed specifically to minimise bias

Is the equivalent of a controlled experiment in basic science

If properly designed it can be free of bias and thus statistically robust

Question 5

What is a randomised controlled trial?

Answer 5

Participants are randomized into 2 or more groups and each group receives a different intervention.

Assignment of treatment occurs “by chance”

Participants and staff both don’t know what they are giving the patient, the treatment or the placebo.

Question 6

What research questions can be addressed by a randomised controlled trial?

Answer 6

Interventional questions; is the treatment effective? Is treatment A more or less effective than treatment B?

Question 7

What are the disadvantages of conducting a randomised controlled trial?

Answer 7

While it has internal validity, having exclusion and inclusion criteria may mean it doesn’t work for people outside this cohort.

Sometimes, can’t be done due to ethical concerns

Can be expensive

Question 8

How is bias eliminated in an RCT?

Answer 8

Blinding of participants and staff

Placebo controlled

Question 9

What is the difference between passive and active placebo?

Answer 9

Passive placebo - mimics administration only

Active placebo - mimics side effects of intervention

Question 10

What is a cohort study?

Answer 10

An observational study where participants are followed over time (Eg effect of birthweight on future development of heart disease)

Participants with specific characteristics are a “cohort”

Differences between them are measured and they are followed over time.

Question 11

What are cohort studies used for?

Answer 11

Determining risk factors/predictors of risk

Aetiology (What causes the outcomes?)

Prognosis (What happens in this disease over time?)

Diagnosis (If test x is positive, what happens to the patient?)

Question 12

What are the pros and cons to cohort studies?

Answer 12

Pros: Best way to identify incidence and relative risk of a disease

Usually a clear view of the exposure-outcome timeline

Can investigate multiple outcomes

Useful to study rare exposures

Cons: Assessing rare events or events that take a long time to develop may be too slow to yield results

Selection bias - hard to match exposure and non-exposure groups exactly

Not a great design for rare diseases

Loss to follow-up can lead to bias (hard to follow everyone in the cohort due to random reasons)

Exposure status might change over time

Question 13

What is a case-control study?

Answer 13

Observational study comparing 2 matched groups of participants; 1 has the outcome, the other doesn’t. The study analyses the differences and identifies risk factors

Question 14

What questions can case-control studies answer?

Answer 14

Risks

Diagnosis

Prognosis

Question 15

What are the pros and cons of case-control studies?

Answer 15

Pros: Most-widely used study design

Simplest to execute

Quick to perform

Cheap

Cons:

More susceptible to bias

Easy to do but can be done incorrectly

Question 16

How is the control group for a case-control study selected?

Answer 16

Control group should be free of the outcome

Represent individuals selected as cases

Selection of the controls MUST be independent of the exposure

Question 17

What are cross-sectional studies?

Answer 17

Measure a population at a particular point in time in order to detect a particular outcome

Question 18

What questions can cross-sectional studies answer?

Answer 18

Frequency - how common is the outcome?

Aetiology - What risk factors are associated with the outcome

Diagnosis - Does the new tst perform as well as the current standard test?

Question 19

What information can’t cross-sectional studies provide us with?

Answer 19

Can’t determine relationship between outcome and exposure

All that can be said is that at a particular point in time obese people had a higher prevalence of arthritis but we can’t make a definitive relationship between arthritis and obesity.

Question 20

What are qualitative studies?

Answer 20

Name given to group of designs which focus on patient oriented outcomes

Qualitiative studies are very systematic in their approach but they also ask questions like:

Why do people….
What are the reasons for…
How do people feel about….

They use interviews, focus groups, and participant observation.

Question 21

What type of strategies are used for qualitative studies?

Answer 21

Involve explicit sampling strategies and systematic data analysis

Portray the voice of the participant, usually absent from qualitative studies and therefore provides complementary information

Question 22

What is the value of qualitative studies if they are so subjective in nature?

Answer 22

There is value in understanding personal experience and how it impacts the patient and practitioner experience

These methods can bridge the gap between scientific evidence and clinical practive and provide additional improvements to patient care.

Question 23

What kinds of problems can qualitative studies help solve?

Answer 23

Patient compliance/adherence with medication

Disparities between how doctors and patients view medical conditions and treatments

Question 24

What is a case study or case-series study?

Answer 24

A study describing a patient or small group of patients affected by a specific disease or exposed to a factor

Question 25

What are case-studies used for?

Answer 25

To create hyptheses and lead to larger incidences of research.

Question 26

What information does good descriptive research provide us with?

Answer 26

Who is affected

Outcome

Why did this outcome occur?

When did the outcome arise?

Where did this happen?

Who, what, when, where, why, how?

Question 27

What are the pros and cons of case studies?

Answer 27

Pros:
One case to initiate a signal

Provide stronger evidence with multiple cases

Observational

Educational

Easy to do

Identify rare manifestations of a disease or drug

Cons:

No control

Difficult to compare different cases

Cases may not be generalizable

Selection bias

Unknown future outcome/follow-up

Question 28

How are case studies used?

Answer 28

They do not rank highly in evidence hierarchy

Least publishable due to small number of patients

but the information can sometimes be important to provide a basis for hypothesis to be built

Question 29

How are study designs chosen?

Answer 29

Based on the type of question being answered:

Example:

(Prevalence is answered with a cross sectional study

Incidence, cause and prognosis are answered with cohort studies.

Treatment effect is tested with controlled trial)

Question 30

What is the hierarchy of evidence for the questions?

Answer 30

Intervention: RCT > cohort > case control > Case series

Diagnosis: Blind comparison to gold standard

Aetiology: RCT > Cohort > Case control > Case series

Prognosis: Cohort > Case control > case series

Prevention: RCT > Cohort study > Case control > Case series

Question 31

How is the hierarchy of evidence ranked?

Answer 31

Using absence of control or comparison groups

Question 32

What is the strongest form of evidence for making clinical decisions?

Answer 32

Clinical practice guidelines which are made using systematic reviews and meta analyses

Question 33

What evidence must we look for to use in clinical decisions?

Answer 33

The best evidence to use in decisions is the evidence
highest in the hierarchy. Evidence from a lower level
should be used only if there is no good randomised
controlled trial to answer a particular clinical question

Question 34

What is the hierarchy of evidence based on?

Answer 34

Risk of bias

Question 35

What is the NMHRC hierarchy of evidence levels?

Answer 35

I = evidence from systematic review of relevant RCT

II = Evidence from at least one RCT

III-1 = Evidence from well-designed pseudo-randomised controlled trials

III-2 = Evidence from comparative studies non-RCTs, cohort studies, case-control studies

III-3 = Evidence from comparative studies (no controls)

IV = Evidence from case series, case reports

Question 36

What are some important challenges to the hierarchy of evidence?

Answer 36

Study design as a definition isn’t always a reliable indicator of the level of risk of bias

There should be flexibility to upgrade or down-grade evidence levels based on quality of evidence

Instead evaluating credibility of systematic reviews and meta-analyses is a better way to use them

Question 37

How should RCTs be considered?

Answer 37

Although it is important to understand the strengths
and limitations of both RCTs (efficacy studies) and
observational studies (effectiveness studies), none of the
study designs should be considered in isolation since all
types of evidence rely primarily on the rigour with which
individual studies were conducted (regardless of the
methodological approach) and the care with which they
are interpreted