Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Statistics PSY3048 > W10: Systematic Reviews and Meta-Analysis > Flashcards

W10: Systematic Reviews and Meta-Analysis Flashcards

1
Q

What is a systematic review?

A

A research project that seeks to give an overview of the totality of research into a specific topic

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Why do we need systematic reviews? (4)

A
  • Vast literatures
  • Primary studies are often contradictory
  • Hard to identify small but important effects
  • Narrative reviews ( Not replicable, prone to bias, can still be contradictory)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Quality of research pyramid

A
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Systematic review: Searching the literature (2)

A
  • Attempting to get all avaliable evidence (including “grey literature and foreign language studies)
  • Will require multiple electronic databases, hand searches of key journals, reference and cited by lists, contacting stakeholders, research instititues and authors
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What is grey liteature? - (2)

A
  • Typical academic literature is published in peer-reviewed literature
  • There is a lot of stuff that does not get published in peer-reviewed literature like researcher discussing preliminary findings in a study presented in a conference (grey literature)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Systematic review: we search literature and define search stratgery based on

A

PICOS framework

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

PICOS framework may not be relevant to

A

observational studies

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

PICOS framework - Population

A

– e.g., offenders, students, patients with depression

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

PICOS framework - Intervention

A

CBT, rapport

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

PICOS framework - Comparator

A

usual care, interrogation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

PICOS framework - Outcomes

A

e.g., hallucinations, information provided

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

PICOS framework - Study design

A

RCTs, longitudinal studies, field studies

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Systematic review - Search literature using Boolean logic
AND

A

Captures all the relevant PICOS elements

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Systematic review - Search literature using Boolean logic
OR

A

To remain specific relevant evidence with each element

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Systematic review - Search literature using Boolean logic
We check stratgeries of similar reviews to

A

improve on search stratgery

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Systematic review - Search literature using Boolean logic
Don’t neglect - (2)

A

regional differences and time-period differences
e.g., Dyspraxia (UK), Developmental Coordination (USA)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Systematic review: How to find grey literature? (3)

A
  • Ask prominent authors
  • Conference proceedings
  • Research group websites/charities
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

When searching literature with systematic review we should document
why should we document (2)

A

everything (e.g., how many articles were identified, how many were duplicates)
This is for transparency when things go wrong and need to be able to reproduc precisely

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Systematic review study selection: Based on inclusion and exclusion criteria all articles would be screened by

A

title and abstract by two individuals and then by full text.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

Systematic review: Study selection - the inclusion criteria should possibly link with the

A

PICOS criteria

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Systematic review: Study selction: Still document everything (3)

A
  • Number of articles identified and excluded at each stage
  • Reasons for exclusion
  • Include a flowchart of studies
22
Q

All relevant information would be recorded (After study selection, inclusion, exclusion)

A

recorded on data extraction forum

23
Q

Systematic review - extracting the data
Full systematic review require two reviewers to (3)

A
  • Permit rapid data extraction
  • Facilitate fast comprehension of key points
  • Contain enough information for synthesis
24
Q

When recording the data in systematic review we want (4)

A

generic study info: author, year, journal
Sample info: nationality, occupations, age
Methods: study design, procedure, setting, length of follow up
Outcomes: effect size, measures of spread, sample sizes

25
Q

When recording data in systematic review what do we do for missing data? (2)

A

Contact authors
Sensitivity analyses

26
Q

Individual studies have flaws, bias leads to effects being

A

over or underestimated

27
Q

Sometimes there is an obligation (in systematic review) to

A

assess the risk of bias in studies

28
Q

What an issue of measuring study quality scales in systematic review? (2)

A

Study quality scales are generally poor
Summary scores are not particularly informative

29
Q

Many systematic reviews do not include

A

statistical combiniation

30
Q

Systematic reviews when it comes to data synthesis tabulate details from all studies to assess a body of evidence by doing (2)

A

thorough analysis of differences between studies
e.g., effects of different samples, measures, settings

31
Q

What is meta-analysis?

A

Mathematical combination of study results to make a combined estimate of population effect

32
Q

Some articles may have effect size in Pearson’s r and another in Cohen’s d
THne we need to

A

Convert to the same metric

33
Q

Meta-Analysis weighted average of all study estimates
e.g,, Group 1 had average 34 test score and 20 people in it
Group 2 had average 36 test score and had 30 people in it

Calculate weighted average

A
34
Q

2 different ways we do meta -analysis (weighted by precision ; SE)

A

Fixed effects
Random effects

35
Q

Fixed effect formula

A

w = 1 / (se)^2

36
Q

Random effects formula

A

w = 1/ (se)^2 + t

37
Q

Fixed effects MA (meta-analysis) assumes

A

all variability between studies is due to sampling error

38
Q

Random effects (MA) assumes that there is

A

sampling error + genuine differences between study populations

39
Q

We would use random effects when there is for example (2)

A

Often times greater hetrogenity than sampling error is some studies use questionnaire 1 while others use different questionnaries

1st questionnaire find bigger effect size than seond questionnaire

40
Q

In psychology, we usually use random effects unless (2)

A
  • You are analysing direct replications
  • You have a very small numberof studies with very different sample sizes
41
Q

We have another metric named ‘t’ (tau) in random effects analysis as it is

A

extra variance beyond that accounted for by study imprecision, i.e.,
between study error

42
Q

New development of instead needing systematic reviews and meta-analysis

A

Massive multi-lab studies running the exact same procedures,
E.g., Psychological Science Accelerator

43
Q

Advantages of multi-lab/multi-site replications over meta-analysis - (2)

A

Low heterogeneity
No publication bias

44
Q

Disadvantages of multi-lab/multi-site replications over meta-analysis - (2)

A

Sometimes the heterogeneity is the important bit
Even narrower in scope, so limited to the most important (and simplest?) research questions

45
Q

Heterogenity analysis aim in random effects is to

A

find sources of systematic variation

46
Q

Aim of heterogenity is to specifically identify two things (2)

A
  • Identify reliable and unreliable measures
  • Identify differences due to samples
47
Q

What is heterogenity analysis done via? (2)

A
  • Subgroup analyses
  • Meta-regression
48
Q

Issues of heterogenity analysis - subgroup and meta-regression - (2)

A
  • Tend to have lower power (both require at least 10 stuides)
  • Multiple comparisons - specificy and justify comparisons in advance when possible
49
Q

Subgroup analyses steps - (2)

A

Divide studies according to a particular feature (e.g., students vs general population, study designs like prospective vs retrospective and measures of DVs)
Run analyses separately - any differences in outcome?

50
Q

Meta-regression (2) steps

A

Linear regression weighted by study precision
Assess impact of explanatory variables upon estimate of effect size (E.g., mean age, length of follow-up)

51
Q

Asymmtery in funnel plot (2)

A
  • There is a gap of missing studies in plot
  • Often what happens when publication bias happens as studies who don’t get effect don’t get published
52
Q

Suppose that we did not have a suitable estimate of the standard
deviation available. Explain how a systematic review could be used
to obtain one (5)

A

In a systematic review we would consult multiple electronic databases,
searches of key journals, reference and cited by lists, and contact stakeholders to identify works reporting standard deviations
of 400m times.

The PICOS framework would be used and searches made via suitable Boolean logic, e.g., “standard” and “deviation” and “400m”.

Based on inclusion and exclusion criteria all articles would be screened
by title and abstract by two individuals and then by full text.

All relevant information would be recorded on data extraction
forms.

The critical details of the studies, including the standard
deviation estimates would then be tabulated to provide the body
of evidence for selection of a suitable value.

Statistics PSY3048 (10 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions