Scientific Investigation Flashcards
5 Steps of scientific investigation (Kazdin, 2003)
OSCAR
- Originate (formulate) a testable hypothesis
- Select a research method and design the study
- Collect the data
- Analyze the data
- Report the findings
O: Originate (formulate a testable hypothesis)
- hypothesis
- null hypothesis
- alternative hypothesis
- IV
- DV
hypothesis
a tentative statement about the relationship between variables
Null hypothesis (Ho)
hypothesis that specifies that there is NO difference between conditions or groups in the experiment on the dependent measures of interest
Alternative hypothesis (Ha)
hypothesis that specifies that there IS a difference between conditions or groups in the experiment on the dependent measures of interest
IV
variable (construct, experimental manipulation, intervention, factor) that is manipulated in the study
DV
the measure designed to reflect the impact of the IV, experimental manipulation, or intervention
S: Select a research method and design the study
- true experiment
- quasi experiment
true experiment
– must meet conditions (Kazdin, 2003):
random assignment of participants to conditions
maximum control of IVs/condition of interest
investigator can include alternate conditions (i.e., treatment and control conditions)
investigator can control possible sources of bias within the experiment that permit the comparison of interest
i.e., randomized control trial is a good way to see if an intervention is effective
quasi-experimental
conditions of the experiment are approximated; restrictions are placed on some aspect of the design (Kazdin, 2003):
e. g., can’t use random assignment to groups e. g., convenience sample
statistical significance
- The criterion is used to evaluate the extent to which the results of a study (e.g., differences between groups or changes within groups) are likely to be due to genuine rather than chance effects.
- The level that demarks statistical significant (called alpha and designated with a Greek letter α) is completely under the control of the researcher
- Norms for different fields exist
- For example, .05 is generally used in educational/psychological research.
What does .05 mean?
- The level of statistical significance is the level of risk that the researcher is willing to accept that the decision to reject the null hypothesis may be wrong by misattributing a difference to the hypothesized factor, when no difference exists.
- In other words, the level of statistical significance is the level of risk associated with rejecting the null hypothesis.
Selecting α = .05 indicates that the researcher
is willing to risk being wrong in the decision to reject the null hypothesis 5 times out of 100, or 1 time out of 20.
Type I error
Decision: Reject the null hypothesis
Reality: there is no significant effect
*probability of rejecting the null hypothesis when it is true (α)
False Positive
Type II error
Decision: Fail to reject the null hypothesis
Reality: there is a significant effect
*probably of accepting the null hypothesis when it is false (beta)
False Negative
Power
- Probability of rejecting the null hypothesis (that there are no differences) when, in fact, that hypothesis is false
- Alternatively, detecting a difference between groups when, in fact, a difference truly exists
- Put another way: the extent to which an investigator can detect a difference when one exists
Power analysis
- a way of deriving at the ideal sample size number
- to increase power, increase sample size
Four concepts involved in power analysis
SEPS
- statistical significance
- effect size
- power
- sample size
*once three are known, the fourth can be determined
Effect size
- The measure of the strength of the relationship
- Cohen’s D
_____ small effect size
_____ medium effect size
_____ large effect size
(Cohen, 1998)
.2
.5
.8
Common measures of effect size
- Cohen’s D
- Pearson’s r
- eta-squared and partial eta-squared
Three different types of control group
- no treatment control (unethical in psychotherapy research)
- waitlist control
(group is withheld from treatment for a period of time, usually time it takes treatment group to complete pre-test, treatment, and post–test)
while waiting, the control group completes pre-test and post-test - nonspecific treatment control
(substitute of pseudo-intervention that involves subjects in some sort of treatment experience (not considered a mechanism of change)
5 types of data that can be included in research
IPPAD
- Interview
- Physiological data
- Psychological report
- Archival data
- Direct observation
Continuous vs. categorical variables
- continuous: interval and scale
- categorical: dichotomous, nominal, ordinal
Effectiveness study
actual patients, actual population
Efficacy study
RCT, random sampling from population, discreet dx
Validity: intent to treat analysis
- Keep everyone in, but you use the last score of those who attritioned for the data
- Can also do chi square (like a T-test, except with nominal variables) on demographics to compare the people who completed vs. those who didn’t to account for more systematic variance
- Intent to treat is contrasted with completer analysis (or per-protocol analysis), which only takes into account those who complete the study. Subjects who have not completed the measures (who dropped out of treatment before posttreatment or follow-up assessment) are omitted from the data analysis. However, this approach does not show the practical value of what is being measured (e.g., a new drug)
What kind of test can you run to double check random assignment?
Chi-square
Drop-our rates of therapy research
- Men (Leong & Zachar, 1999)
- Low SES more likely to drop-out (Leong & Zachar, 1999)
- Ethnic minorities are more likely to drop-out (Leong & Zachar, 1999)
- People over 65 (Leong & Zachar, 1999)
Luborsky’s Therapist Allegiance Effect (Luborsky et al., 1999)
- whatever orientation of the researcher tends to be the one that is significant
- APA – in order for therapy to be considered evidenced-based, research needs to be conducted by at least two independent researchers
anytime RCT is used, researchers must
randomly videotape to adhere to APA guidelines
When doing an experiment or quasi-experiment, you always have to run ____
a manipulation to check to see if your variable changed anything
Pearson correlations can only be used with what type of data?
interval and ratio
Kendall’s Tau (correlation) is uses for
non-parametric (nominal) data
