Lecture 1 - Systematic Reviews, Meta analyses, and clinical practice guidelines

Question 1

What kind of review is when the author selects the articals that back their perspective?

Answer 1

Narrative Review

NOTE: there can be lots of bias w/ this because they can only take one kind of articales

Think of this as a movie review, where someone is summing up the show/episodes and what the plot lines were and what they genually thought

Question 2

What kind of review has a rigorous process of searching, apprasing and summarizing articals. Objectively chooses sources. Follows prisma checklist to select stuidies. Allows you to get all sides of the problems. Comprehensive report of research findings.

Answer 2

Systematic review

Think of a systemiatic review as a critical appraisal of that show. How they did the lighting / senary. what the themes were in the show.

Question 3

Pools data of several simlar studies together (probs use the same outcome measure) and are typically systematic review studies. What is this?

Answer 3

Meta analysis

NOTE: we would do a systematic review when comparing studies where the outcome measures are different

Question 3

Systematic review:
* Provides a compreshensive and objective summary of all relevant studies on a specific research question, following a standardized and transparent process
* Highly strucutred and systematic. Inolves a predefined protocal, including clear critera for study selection, data extraction, and analysis
* Minimizes bias through rugorous methods, including efforts to include all relevant studies and appraise their quality
*

Question 4

Narrative Review
* Purpose: Provides a broad overview of a topic, often summarizing existing literature w/o following strict methodology
* Selection of studies and sources is typically based on the authors experties and perspective
* Prone to author bias since the selection of studies is subjective and not necessarily comprehensive
*

Question 5

Meta Analysis:
* A statistical technique used to combine the results of multiple studies identified in a systematic review to produce a quantitative summary of the evidence
* Involves statistical methods to aggregate data from several studies, often using effect sizes, to provide a more precise estime of the effect or association being studied
* Reduces bias by pooling data from multiple studies, but the quality of the meta-analysis depends on the quality of the included studies
* Provides a staticially driven conclusion, offering a more robust estimate of the eect or relationship being studied

Question 6

Systematic review of the literature as put out as a statement of recommendation by professionals in whatever setting that is. Not just a set of guidelines for PT’s but also for other practies as well
* Systematically developed sattement designed to faciliate EBP

Based on a specific diagnosis (so what should we do w/ these people - all the way from prognosis and disease course, all the way to discharge critera and everything inbetween)
* Chronic LBP
* Adhesive Capsulitits
* Patellofemoral Pain

Answer 6

Clinical Practice Guidelines

Question 7

Why do we do systematic reviews?

Answer 7

To summarize current evidence on a stopic in a systemiatic way
* Saves the audience time.

NOTE: They’re also nice because they can look at any and all parts of a physical therapists care, including:
* Interventions
* Prognosis/Risk Factors
* Develop recommendations for CPG’s/practice - often based on systematic review
* Diagnostic Accuracy - how good our special tests are etc…
* Qualitative Research - best summarized by a systematic reviews (better than a meta anaylsis)

Question 8

NOTE: Its very hard to create quantitiative data out of qualitittive data. We attempt to do this by outcome measures that are questionaires that rate things 0-5. Were rating qualititive things in a quantitiative way.

Question 9

What summarizes qualitative materal better, a systematic review or meta analysis

Answer 9

systematic review

Question 10

NOTE: Systematic reviews are useful when multiple RTCs have been done, but report different findings.
* Systematic reviews systematically gather and analyze all relevant RTCs on a specific question, allowing researchers to compare and contrast the findings in a structure way. This process helps identify patterns or reasons for the discrepancies between studies

Question 11

KNOW: Inclusion and exclusion criteria are specific guidelines used in researcj, espeically in systematic reviews, to determine which studies should be included or excluded from the review. These criteria are established before the review begins to ensure consistency and reduce bias in the selection criteria

Question 12

Characteristics that a study must have to be included in the systematic review?

Answer 12

Inclusion Criteria

Purpose: To ensure that the studies included in the review are relevant and provide meaningful data for answering the research question

EX: Studies that focus on a specific population, such as adults w/ type 2 diabetes

EX: Studies that investigate a particular treatment or intervention, like a specific drug or therapy

Question 13

Characteristics that disqualify a study from being included in the systematic review

Answer 13

Exclusion criteria

Purpose: To exclude studies that are irrelevant, of low quality, or do not directly address the research question

EX: Studies that focus on a different population, such as children instead of adults

EX: Studies that examine interventions not relevant to the review, like a different type of therapy

Question 14

Inclusion/exclusion criteria for studies based on:

Answer 14

1) Study design
2) Participants
3) Topic of interest
* Interventions
* Risk factor
* Outcome measure
* Diagnostic Test

Question 15

Variable that the researcjer manipulates or changes in an experiemnt. Its the cause or factor that is being tested to see its effect ont he dependent variable

Answer 15

Independent variable

It is the “input: or “predictor” variable, and its variation is what the researcher is interested in studying

EX: if you’re testing the effect of different amount of sunlight on plant growth, the amount of sunlight is the independent variable. The researcher changes the sunlight levels to observe the effects.

Question 16

The dependent variable is the variable that is measured or observed in response to changes in the independent variable. It’s the “output” or “effect” that depends on the independent variable.

Answer 16

Dependent variable

Its is the outcome that the researcher measures to determine the impact of the independet variable

EX: In the same plant growth experument, the Frowth of the plant (measured be height for instance) is the dependet variable. It’s what changes as a result of the different levels of sunlight

The independent variable is what you change, and the dependent variable is wjat you measure. The relationship between them is central to understanding cause and effect in experiements.

Question 17

Systematic review: (This entire thing is a systematic review - which compares other studies)

Looking at athletic concusion

Dependent variable = return to sports time
Independent variable = interventions

We want to look at what interventions decrease return to sport time the most.

Now when looking for studies we want to decide which inclusion / exclusion critera to use
* Study design: RCT = highest quality to answer this question –> RTC –> Cohort –> Case Control Studies –> Case report or Case series
* NOTE: Their initial inclusion/exclusion could be just wanting RTC’s however, the might not get many w/ just that and have to include cohort etc…
* But just know, if their includsion criteria include those lower ranked studies this is going to be a lower quality study. We want to see if they address this: “We were forced to use case report/case series because there just werent enough RTC’s”.
* Participants: We want to make sure partipants in these studies that we include match their population.
* If they did a randomized study that did concusion, but was in the elderly - well thats not answering our question
* We want to know that all the participants are very similar and if they match the intention of their focus
* Topic of interest: Were specifically interested in interverventions - so all the studies that they include, their independent variable (thing thats changed) so should some kind of intervention
* If they include a study thats “return to sport risk factors” than thats not an intervention
* Whereas if it was Q angle greater than something vs Q angle less than something thats not modifiable, thats not an intervention - were specifically looking for some intervention
* We want to make sure the studies they’ve used matched our intention.

We want to look at the databases they’ve utilized. If they only use 1 database, thats a red flag. We want to see a varierty.
* Most will use 3
* Will use 1 priamrily and the other 2 as a check

Search Strategies:
* Search terms: make sure search terms match their intentions - so if they’re searching concusion in adolesen but nothing about sport - well this doesnt match what we want
* Restrictions based on languge/full text aviable - well if they’re only searching for english that can add bias because that might be different that the level of knowledge in china etc…

Screening references:
* How did they get rid of duplicates?
* How did they determine relevance?

Data Extraction:
* Once they decided the study was relevant and to be used what parts of that study did they then extract out

Question 18

Rank the different studies in order

Answer 18

1) Clinical Practice Guidelines
2) Meta Analysis / Systematic review (a meta anaylsis is a kind of systematic review)
3) Randomized Control Trial
4) Cohort Studies
5) Case Control Studies
6) Case Report or Case Series
7) Animal and Laboratory Studies

Question 19

PRISMA flow diagram

Inital search was 3092, they had 579 duplicates, leaving them with 2513. They then exluded 2323 because they werent releated (must list the reasons why they were exlcuded [like done below]). Full text articules (because you don’t have acess) - then showing why some of those were excluded.

Then they finally have 9 studies that match (this is a typical progression)

And then how we would asses the quality of each study: PEDro scale / Cochrane risk of Bias (best for RTC intervention studies).

PRISMA flow diagram:
* Pereferred reporting items for systematic reviews and meta analysis
* visually represent the process of selecting studies for inclusion in a systematic review/meta
* It shows the flow of information through the different stages of the review process, from identifying studies to including / excluding them

Question 20

What is best for asses the quality of RTC internvention studies?

Answer 20

PEDro Scale
Cochrane Risk of Bias

Question 21

KNOW: The Pedro scale is specifically used to asses that quality of physical therapy articales, however the Cochrane Risk of Bias is most broadly used in all professions.

For our projects we are going to have to fill our the Cochrane Risk of Bias, or the PEDro scale if youre doing a study about interventions are utilizing RTCs

Question 22

Studies of Diagnostic Accuracy = Quality Assessment of Diagnostic Accuracy Studies (think how good a special test is at diagnosis something)
* Would use a QUADAS assesment (to asses the quality of the study?)

Question 23

What scale of quality is best for studies of diagnostic accuracy?

Answer 23

Quality Assesment of Diagnostic Accuracy studies (QUADAS)

QUADAS assesses the methodological quality of diagnostic accuracy studies, ensuring that the studies are designed and conducted in a way that procides trustworthy results.

Question 24

What scale of quality is best for studies of prognosis?

Answer 24

Quality in Prognosis Studies (QUIPS)

Would be appropriate for cohort and case control studies

Examines 6 sources of bias

Prognostic studies: assess the likelihood of a specific outcome for a group of people based on their clinical and non-clinical charcteristics

Question 25

What scale is best for studies of clinical measurement? (studies to test to see if outcome measure is good and other stuff as well I think)

Answer 25

Consensus-based standards for the selection of health measurement instruments (COSMIN)
* Checklist
* Patient reported outcomes

Focus on assessing how well clinical tools, tests, or instruments measure health care outcomes.
* EX: Might examine whether a blood pressure monitor consistently gives acurate readings or whether, a new diagnostic test correctly identifes a disease.

Question 26

What scale is best to assess the quality of observational studies?

Answer 26

Newcastle-Ottawa Scale Quality Assesssment Form (NOS)

Question 27

KNOW: For a systematic review its best to keep question open ended so you’re not excluding things that don’t match your theory.
* Were asking which intervention is best
* We want a lot of studies with a lot of different interventions to figure out which one is best to change a specific outcome
* Different than a RTC where we have a hypothesis that our intervention is better than placebo or other interventions
* This is an independent objective review of the evidence

Question 28

Which two scales are the best for assesing RCTs w/ interventions?

Answer 28

PEDRo Scale/Cochrane Risk of Bias

Question 29

What is the PRISMA Flow Diagram?

Answer 29

Process to identify final selection of studies reviewed

Question 30

What is an extension of a systematic review that mathematically analyzes pooled data?

Uses multiple studies to generate a larger participant population

Increased ability to detect important differences/similarities

Answer 30

Meta Analyses

The legos represent different studies, a meta analysis allows us to compare the different studies together, giving more weight to the studies that have more participants etc…
* Can help us fill in the gaps in data: some places in 1 study might be missing data but can be filled in by another studies data

Question 31

Assesses if findings change if key assumptions differ

Answer 31

Sensitivity analysis

If no changes are made, then we can assume that data is pretty strong

We want to check and see if they did a sensitivity analysis and if any substantial changes occured

In other words, if changing important factors doesnt affect the outcome, you have more confidence that the data and conlusion are robost.
* think trying a drug for adults only then including kids and it works the same.

Question 32

Variation of individual study effect sizes (how spread out data sets are) (i think this is talking about comparing 2+ different studies)
* what can this be due to?

Answer 32

Heterogeneity

Due to:
* Population (size, participants) - a study w/ 21 people vs 300. if we found a significant result w/ 21 but not 300, than we should probs lean toward the 300 person study.
* Interventions (protocol)
* Outcome measures
* Study design

When a study shows heterogeneity, it means that the effects found in one study might be larger, smaller, or even in the opposite direction comapred to others. This variation could be due to differences in study design, particiapnts, methods, or other factors.

EX: one study might show a large effect of treatment, while another shows a smaller effect or no effect at all. this variation could be due to differenes in study populations, methods, or seetings.

Heterogeneity indicates that the findings across different studies are not identical and that facors like study design or sample differences might be influencing the results

Question 33

Magnitude of the effect the independent variable has on the dependent variable

Answer 33

Effect size

In similar terms, the effect size tells you how much of a difference or change the independent variable causes in the dependent variable. Its a way of quanitifying the size of the effect in an experiement study

its a way to measure how strong the relationship between the 2 variables is in an experiement - how much the independent variable changes the dependent variable

EX: A new drug might slightly reduce symptoms, but if the effect size is small, the change might not be noticeable or woth the side effects

EX:
* Indpendent cariable: A new teaching method
* Depdent variable: Student test scores
* Effect size: If the effect size is large, it meants the new teaching method significantly improves test scores. If it is small, the new method doesnt amke much of a difference compared to the old one

How much the independent variable affects the dependent variable

if our study finds there is a significant difference between groups, this lets us see how big that difference is

NOTE: Effect size is often weighted depending on the study it comes from:
* Study quality
* Population size

Question 34

KNOW: One way the effect size is documentated is the standarized mean difference (SMD)
* basically this puts it in terms of standard deviations
* Difference between individual study group means divided by their pooled standard deviation
* on average how many standard deviations away from the mean is that data

Answer 34

The mean si the avearge score of the group

Standard deviation: a measure of how spread out the scores are around the mean

Standardized mean difference: were saying “on average, how many standard deviations away from the mean is one group comapred to another”

EX:
* Group A has treatment, group B doesnt
* if the SMD is 1, it means that group A’s scores are 1 standard deviation higher than group B on average

This standardization allows us to compare effects across studies, even if they used different measures or scales
* 0.2 = small effect size
* 0.5 is a medium one
* 0.8 or more is a large effect size

across these studies, not matter the scale. Because SMD exoresses the difference in terms of standard deviations not raw points

Question 35

KNOW: The standarized mean difference (SMD) is a specific way to express effect size, particulary when comparing the means (averages) of two groups. It tells you how much one group differs from another in terms of their average scores, taking into account the variability within groups

The standarized mean difference is a specific way of expressing effect size that allows you to compare the magnitude of differences between groups, regardless of the scale of measurement. It gives you a clear, standardized measure of how much one group differes from another, helping you interpret the practical significantce of your results

Question 36

what is this?

Answer 36

Forest plot

A forest plot is a graphical representation used in meta-analyeses to display the results of multiple studies on the same topic, typically comparing the effects of an intervention or treatment. It visually summarizes the individual study results and the overall combined effect

Each study included in the meta-analysis is represented by a horizontal line and a square or dot on the plot

Horizontal line: This line shows the range within which the ture effect size is likely to lie for each study, typically using a 95% confidence interval

Square/Dot: The square or dot represents the point estimate of the effect size for the study. The size of the square may vary depdening on the study’s weight or same size, with the larger squares indicating more influence on the overall result

At the bottom of the forest plot, a diamond shape typically represents the pooled effect size, which combines the results of all the studies

Diamond width: The width of the diamond shows the confidence interval fro the combined effect, indicating the precision of the overall estimate

In the aboe they all seem to favor the intervention

The line going through each boxes shows the 95% confidence interval
* A wide one = bigger range

Question 37

interpreting this is going to be an exam question

Theres no change in risk, theres a slight increase in risk etc…

And then how would you educate the pt on what to do could be an open ended question she would include

Question 38

What does this line being wider mean?

Answer 38

Increased variability

This line represents the 95% confidence interval

Question 39

What kind of a plot is a funnel plot?

Answer 39

A scatter plot

Question 40

What does the solid line in the middle represent in this funnel plot?

Answer 40

The average

Question 41

overall effect = the average (so the average is a slight increased risk in this study)

So the studies found on average a slight increased risk. The ones to the right showing a more increased risk and the ones to the left showing a less increased risk (0 would be no increase and no decrease)

Question 42

The standard error is on the left side. What is it saying?

Answer 42

Its essentially letting us know the quality of the study
* Systematic review vs RTC etc..
* So we would think that those good studies (meta / systematic review) are going to be at the top meaning they have less error

Case controls and other poor study types / studides w/ a very wide confience interal (much variety within that 95% confidence interval) are going to be lower because they have more error.

Question 43

NOTE: w/ funnel plots we want to see some high quality studies, some low quality studies and they’re all grouped nicely around that center line
* NOTE: typically theres going to be more lower quality studies (higher quality studies are much harder to do - and a meta analysis / systematic review needs lots of those lower level studies (RTCs/Corhort etc..) to complie, so we litteraly cannot have as many higher level studies)

Question 44

NOTE: the dotted line represents the 95% confidence interal lies somewhere within that graph space.
* So the higher quality ones should lie closer to the average and not gave as much of that deviation (which is why it gets closer at the top)

Question 45

Knee OA is our diagnosis

Risk factor = age > 60

We will now take a bunch of studies that look at this data (age vs knee OA)

Systematic review
* Relative Risk = 2
* 95% confidence = (1.2, 3) (were 95% sure that with this risk factor they fall within this much relative risk)

Meta analysis
* relative risk = 2.5
* 95% confience = (1.01, 4.1)

Cohort
* relative risk = 3.2
* 94% confidence = (2.1, 4.3)

Cohort
* relative risk = 1.2
* 95% confidence = (0.48, 1.5)

Now the funnel plot just puts them on a graph

So the higher quality studies go higher up on the graph (less error)

We take the average of the relative risks to find the avergae of the graphs = 2.25

We take the average of the 95% confidence intervals to find the bottom 2 points of the funnel plot (just picked numbers below not accurate)
* 0.48
* 4.3

Note we use log at the bottom because that essentially puts it into a universal unit (not shown on my picture below)

Note: MA = top on the graph (found slighly above so its to the right)
* SA found lower than average so its to the left
* Lower quality ones are lower

NOTE: the lowest quality artical 95% confidence interval is 0.48, and 4.3 which is the base of the pyromid and at the bottom.

Question 46

Relative risk

Answer 46

A measure of the risk of a certain event happening on one group, compared to the risk of the same event happening in another group

A relative risk of 2 means that a person is 2 times more likely to have that diagnosis w/ that risk factor

Question 47

KNOW: we want the dots to fall within the funnel, its okay if a couple of the lower ranked studies fall outside but most of the studies should be in.
* we also want a good variety of studies, we shouldnt just have high quality studies or just low quality studies
* we should have more low quality studies than high quality studies
* we also want the studies to fall on both sides of the average line (not just one outliar study that pulls it way one direction)

Question 48

Is there bias here?

Answer 48

Yes, we have 2 high quality evidence studies and one is outside that 95% confidence interval

The average is skewed (lots of studies to the right of the average line)
* the mean is pulled to the left by the outliar

we do have a good ratio from high quality to low quality studies though

Question 49

Is there bias in this funnel plot on pain?

Answer 49

yes

lots of studies outside that 95% confidence interval

most studies are to the right of the average

Question 50

is there bias in this study

Answer 50

No

Question 51

Effect size = how much the independent variable changes the dependent variable

Effect size 0 = independent variable not affecting depdent variable

Effect size negative = indepdent variable is decreasing depdent variable?

Effect size positive = indepdent variable is increasing depdent variable?

Question 52

Questions we want to ask ourselves when reading a systematic review or meta analysis

Does the systematic review inform clinical practice releated to my clinical question? - does it add to whats out there, or is it just restating something thats already been done?

Was the literature search comprehensive? - meaning were their search terms fully inclusive of the topic that they’re studying, did they use a variety of data bases?
* Seach terms
* Databases
* Languages

Was an objective, reproducible and relaiable method used to judge the quality of the studies? - if someone else wanted to do your study the exact same way, could that do that?
* PEDro/Cochrane/etc

Was a standard method used to extract data from the studies? Meaning, if im doing a systematic review and these are my studies, how am i exacting this data? am I using studies w/ the same outcome measures? if not how am i extracting that data

Was clinical heterogeneity assessed? Meaning how similar were the studies
* This is going to say, was a meta analysis justified
* if all the studies used different outcome measures (all measured different risk factors etc…) than a meta analysis might not be best
* If heterogenity is low - meaning they’re much more similar that would be a good reason to do a meta analysis

How was the data reported
* Systematic review = some kind of a table or visual
* MA: Table, graph, forest plot, funnel plot, something to show all the data
* If you’re looking at a statistical analyses which ones were performed? and does that match the purpose? If were looking at a risk factor, doing a relative risk or an odds ratio makes sense.

Question 53

Clinical Practice Guidelines
* Informs medical practitioners in their evidence based practice (not just for PT’s but lots of professions)
* Does outcome measure, prognosis, clinical course, etc..
* They inform practice

Combines all parts of EBP
* Evidence
* Clinical expertise
* Patient perspectives

Comrehensive (covers all this)
* Evaluation/Examination
* Prognosis
* Treatment

Databases that house CPGs
* Nice
* PEDro
* PubMed

Journal for PTs
* Journal of Orthopedic and sports physical therapy

pictures on both pages showing a CPG for physicians and BP

Answer 53

Question 54

CPGs: Quality Appraisal
* Note: Not all CPGs are perfect, always some level bias

What are the two questions we need to ask ourself for these?

Answer 54

Who is involved in providing information and/or creating the CPG?
* was it just one peron or was it from many different angles
* Contribuitors to CPG - fully representative
* Patient preferences included - we dont know what the patients expectations for outcome was
* Target users defined? was it for PT’s or surgens etc…

What was the rigor of their methods?
* Systematic methods utilized for?
* Critera for selecting evidence defined?

Question 55

CPGs: The results
* This is typically waht were reading as PT’s
* Recommendations
* Intro
* Methods
* Specific guidelines and supporting data
* Summary of recommendations

F = lowest
A = highest - more than 1 strong study (systematic review/Meta) supporting these things

B = 1 strong artical + other weaker ones

D = conflicting

Question 56

CPG Recommendation Levels

Question 57

Critically Appraising CPGs

Answer 57