Meta Analysis Flashcards
Meta Analysis
so what is it?
Meta-analysis is a way of using results from different studies, about the same issue,
and studying them as a whole to look for an overall picture about the area of study.
It is more a technique of an analysis than a research method,
it uses studies that have already been carried out with the research methods chosen and separate analyses already done.
If a number of studies separately find the same answer, and then those studies are analysed together (a meta-analysis) then that answer becomes stronger as the studies support one another.
A meta-analysis can also clarify results where studies find different answers.
For example, if one study finds that pre-trial publicity does not have an effect on jury decision making and
other studies have found the opposite, that might something worthy of a meta-analysis.
to clarify if ptp does or dosnt have an effect in JDM
Meta Analysis
primary data
Primary data is information observed or collective directly from first-hand experience
data that has been collected by the research of the study currently being undertaken
specifically relating to the aims and/or hypothesis of the study
examples of primary data are: the result of experiment or the answers my questionnaire etc
Meta Analysis
secondary data
Secondary data is information that we collected for a purpose other than the current one/study
examples of secondary data are government’s a districts results from another experiment by the same or different researcher ETC
in a meta analysis secondary data will be used to come to a conclusion
Meta Analysis
types of data produced
Meta analysis produces quantitative data via statistical analysis of all of the combined data from different studies (secondary data)
this is to find patterns and relationships between the factors
e.g which defined characteristic has a largest effect on jury decision making: race, accent, sex etc
the main focus is the effects/direction of the factors / Concepts being investigated
not the significance as the original research has studied this already
This data is used to come to an overall decision on a subject for example whether or not pretrial publicity has an effect on jury decision making
Meta Analysis
Data requirements for successful meta analysis
The data from all of the studies included should be taken in a similar format so all of the data is comparable with one another
and that the statistical tests are valid
In a meta analysis you must only include studies that have good methodology
therefore meaning that only the best evidence can be used in the overall analysis
not bad evidence from poor methodology
Meta Analysis
Is meta-analysis useful in criminal psychology
Meta analysis provides insite into:
overall effectiveness of interventions (such as psychotherapy, rehabilitation for criminals etc)
the relative impacts of independent variables (such as the effect of different types of therapy)
and it can also be used by expert Witnesses
as it can show if the data given is an outlier and unusual compared to other studies
and doesn’t represent what field shows as a whole
which helps clear up accusations of age discriminations
apparently
Meta Analysis
Strengths of secondary research
a strenght of secondary research is that secondary data can have zero ethical issues
as the studies that they are from have already taken place
and therefore no ethical guidelines can be breached for the meta analysis
as the data is second hand
Meta Analysis
Weaknesses of secondary research
• data may not be taken in a similar way / design as the others and therefore cannot compare them validly
meaning the meta analysis may be invalid due to these invalid comparisons between sets of data
• not all studies have the same level of control over extraneous variables such as experimental biases or demand characteristics
so an unknown number of secondary data may be invalid
therefore making the conclusion of the meta-in analysis invalid if there is enough invalid data
• participants may not have consented for their data to be used in another study only the original study that they partook in
meaning that this is one of if not the only ethical issue and a meta analysis could form
Meta Analysis
Step by step
- Define the research question aka what are you researching/ looking for
- Perform the literature search, identified relevant studies
- Select the studies from a set criteria of what to include and what not to include
- Extract the data from these studies
- Analyse of a data from the studies using a array of statistical tests for relationships like a spearmans rho test
- Report the results usually in a report/journal
bonus meta analysis things
Meta analysis provides the means too define from a broader database than individual studies.
the presence / absence and magnitude of an area of study
(e.g pressence/absence and Magnitude of the PTP effect)
Meta analysis allows methodical issues to be overcome such as:
• lots of data sets leading to uncovering effects that are overwise Masked by not having enough participants
Meta Analysis
large amount of data IV + and Gen too mabye
Meta analyses collect a large amount of data
for a vast array of different areas getting a huge overall sample very quickly
which is a strength as a large sample reduces the effects of participant effects and data anomalies
allowing for a more valid establishment of causing effect meaning that meta analysis has internal validity
Also due to a larger sample it’s more likely that it will be more representative of groups in the population such as cultures ages etc
Therefore improving the results generalizability
Meta Analysis
Not carrying out any research ethics +
A strength of meta analysis is the researchers are not carrying out any research themselves
therefore avoiding any concern to regards on how participants were treated
therefore meaning that they’re can be no ethical breaches of guidelines
such as deception or protection from harm therefore making meta analysis highly ethical
Meta Analysis
No involvement in Gathering data and Undisclosed methodology called issues may affect conclusions drawn - IV
A weakness of meta-analysis is that the researchers do not have involvement in gathering the data directly
and therefore Undisclosed methodologicall issues (e.g. demand characteristics)
may affect the conclusions drawn these would be classed as extraneous variables
weakening the link between cause and effect and therefore meaning that meta analysis will have a lower internal validity
Meta Analysis
Not include unpublished work + iv
A strength of meta analysis is that researchers may often not include any unpublished work
meaning that all the findings that are drawn upon have been peer-reviewed
improoving the overall validity of the results
as often unpublished data has poor methodology
meaning the data collected from the study is poor and possibly invalid due to a variety of biases such as publication vise or experiment biases
therefore meaning that when researchers do not include unpublished work
the overall results will have less extraneous variables and therefore less than effects on cause and effect meaning that analysis will have high internal validity
Meta Analysis
An application
it can be used to help resolve issues with Juror Decisions being invalid (biased)
A strength of meta analysis is that it Focuses on the size of somethings effect (quantifying differences between two groups)
AKA
in criminal psychology this means that we can establish the impact of ptp on Juror opinions
and thus adapt courtroom procedings to reduce the effect of ptp on juror’s decision making
improving the overall validity of the justice system
Meta Analysis
Allows for reliability +
A strength of meta analysis is that it allows for reliability
as the higher the percentage of studies that find similar results
the more likely it is the results are true
and therefore meaning that the overall result of the meta analysis and those studies used in it have reliability
making them Reliable
Meta Analysis
Objective data + iv
A strength of meta analysis is that it uses quantitative data
which can’t have subjective or debatable meaning in its analysis
due to the data being numerical therefore meaning that whether it is high or low the meaning of this is always clear
As meta analysis uses this type of data the statistical tests it uses to come to a conclusion is also objective
as the cause and effect link between the Factor/ concept/ relationship between factors and it’s effectiveness/effect can accurately be established
Meaning that meta analysis due to its use of objective quantitative data has high internal validity
Meta Analysis
small studies different designs - iv
A weakness of meta annalysis is that the studies that it uses may have different designs
meaning that it may be difficult to effectively compare them as if the data is collected in different ways it may affect the data itself
compared to being collected in just a singular way
this is an extraneous variable which decreases the link between cause and effect lowering meta analysis internal validity
Meta Analysis
Meta analysis may be invalid due to a number of biases - IV
• Experementer Bias: (when the experimenter beliefs influence the results)
• search bias: which is when the search for studies can produce unintentionally biased results this includes using an incomplete set of keywords or varying strategies to search databases also research engine used can be a factor affecting the studies found to be able to be selected in the first place
• Selection Bias: which is when researchers must clearly defined the criteria for choosing from the long list of potential studies to be included in the metronalysis to ensure unbiased results if this is not done then the studies used in the meta analysis maybe bias towards a certain conclusion.
If enough of these biases are present in studies in the metronalysis then the whole meta analysis will have many confounding variables within it meaning cause and effects cannot accurately be established and therefore meaning the metronalysis will have low internal validity
Meta Analysis
Unpublished data used - IV and gen
A weakness of meta analysis using unpublished data is that it may introduce a bias
Unpublished studies that can be located May be of an unrepresentative sample of all unpublished studies done (therefore leading to low Generalisability as Un rep sample to Research as a whole so cant effectivly compare them all)
They may also be of low methodology cool quality compared to publish studies making them difficult to compare them as they may have bad methodology therefore bad data which can lead to invalid conclusion
This is confounding variable that reduces the link between cause and effect leading to an decrease in meta analysis internal
Meta Analysis
not using Unpublished studies - IV ( publication bias)
A weakness of not using unpublished studies no matter analysis is caused by publication bias
Publication bias: this is when positive studies are more likely to be published than studies that find results that go against the majority view of results in that particular field therefore leading to the sample being bias and the Meta analysis not properly reflecting the Field of research as a whole
Meaning that the metronalysis itself will have an invalid resultant conclusion due to not analyzing the Field of research as a whole including possibly negative studies that did not go to print
Meaning cause and effect can’t be established giving metronalysis low internal validity
