Consensus in Neuropsychological Research Flashcards
What are 4 goals of meta-analyses according to Tak et al (“more than the sum of its parts”)?
- By summing several studies their results you can create more power on a specific finding.
- Creating insight in why studies differ in their findings
- Raise the positive predictive power of true relationships
- Identifying areas that need further investigation
How are the results from a meta-analysis graphically presented? And which 3 information points are thereby presented?
À forest plot;
- The results from individual studies
- The amount of variation between studies
- An overall estimate of the results of all studies combined
What is heterogeneity in meta-analysis? And how is this assessed?
This means that the included studies differ considerably on one or more important aspects which may affect the comparability of the results. You assess this to see if the study grouping you have done is actually justified.
Assessed with the Q-statistic (Qtotal = Qbetween +Qwithin)
- -> Qbetween = assumed to be high + significant (indicating heterogeneity); possible need for moderator analysis
- -> Qwithin = assumed to be low + nonsignificant (indicating homogeneity)
What is a moderator analysis?
Performing a meta-analysis on subgroups based on characteristics that potentially are responsible for differences in ES between studies; this can demonstrate whether the strength of the summary estimate is influenced by these characteristics.
What is meta-regression? And how does this work?
This is a regression-based analysis that aims to test for study heterogeneity by associating study characteristics with study outcome.
–> you use this to determine which characteristics should be included in a moderator analysis: as the dependent variable, you use the study’s ES and as the predictors you use the study characteristics (age, gender, ethnicity, etc.). When the predictors are significant, this shows the possibility of being a moderator.
What are sensitivity analyses?
They adress the question whether the findings of the meta-analyses are robust to the methods used to obtain them (f.e. assessing the influence of methodological quality of the included studies, the included entry criteria, etc.)
How is publication bias assessed?
`This is done with funnel plots; visual representation where the effect estimate is plotted against a measure of precision (such as sample size or the SE of the ES). When there is no publication bias > the plot has a funnel-like shape.
Another possibility is the fail-safe-N method: how many new, unpublished non-significant studies are necessary to turn the significant result of the meta-analyses to a non-significant one? When this number is small, the finding is not robust.
What is suggested to be the best way to handle meta-analyses? (According to more than the sum of its parts)
- It is really important to investigate possible causes of heterogeneity
- The statistical combination of studies should not be the main aim of meta-analyses, especially when heterogeneity is the rule rather than the exception
What is a meta-analysis and what is its main goal?
A meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into an ES and then pooling and analyzing this information. Its main goal is to explain why different outcomes have occured in different studies.
What are the different indices of effect in meta-analysis and when do you use which one?
- r –> when both IVs and DVs are continuous
- SMD –> when most IVs and DVs are discrete/categorical while the outcome measure is continuous
- Odds ratio (or one of its variants) –> when 2, but outcome measure is largely categorical
In what range do ES of between-group design interventions fall mostly?
from -0.50 to +1.50
What are the 5 phases of a meta-analysis according to Roberts?
- Formulating the RQ
- Obtaining a representative study sample
- Obtaining info from individual studies
- Conducting appropriate analyses
- Reaching conclusions and guiding future research
Which three models are used to effect size data? And what is their overarching goal?
- Fixed effects models: systematic differences, hoped to be captured by coded variables
- Random effects models: random differences which cannot be identified
- Mixed effects models: combination of unidentifiable random differences and systematic differences that could be identified.
Their overall goal is to determine why different studies produce different effects.
What are the 5 basic steps of the meta-analysis/review procedure?
- Research question: a priori hypotheses, clear question, testable
- Literature research + study selection: exclusion/inclusion, search terms, search strategies
- Performing the analysis (quantitavely): ES per study, weighted, mean overall effect. homogeneity, publication bias, rival explanations.
- Qualitative analysis
- Reporting results
How is the qualitative rating in meta-analysis defined?
The assessment of methodological quality of all studies conform specific operational definitions. In this, you have primary (pp characteristics, IV, DV) and secondary indicators (blinding, random assignment, generalization).
