Research & Program Evaluation Flashcards
Compares two means for one variable; provide a ?-ratio
t-tests; t-ratio
Involve comparing two independent groups (usually assigned randomly) on one dependent variable.
EX: Gender and achievement
Independent t-tests
Involve similar groups paired or matched in some meaningful way or the same group tested twice.
EX: Same group of high school students; pre-test and post-test
Dependent t-tests
A statistical test that involves having at least one independent variable in a study with three or more groups or levels.
EX: Household income (Below $20,000; $20K-$40K; $40K-$60K; and $60,000+) divided into 4 levels/groups
Analysis of Variance (ANOVA)
ANOVA
Indicates 2+ more of the group means are statistically different
F ratio
Used with 1+ independent variable; yields both main effects and interaction effects (i.e., significant differences among groups across or more independent variables).
EX: 2 treatments (Cognitive therapy and Interpersonal therapy) compared for effectiveness on males and females
Factorial ANOVA
Statistical test that includes an IV as a covariate, or a variable that needs to be statistically adjusted & controlled in order to look at the relationship of other IVs & the DV. EX: Examining the relationship bw household income & work satisfaction w/gender as a covariate (statistical effects of gender are removed from the analysis to control for any effects that gender may have on work satisfaction)
ANCOVA
Allows examination of every possible pairing of group means for a particular IV after one has concluded that there are main effects (i.e., significant difference among two or more groups comprising a single IV) in an ANOVA.
Post hoc analysis
Chi-Square test, Mann-Whitney U test, Friedman’s rank test, Wilcoxon’s signed-rank test, Kolmogorov-Smirnov Z procedure, Kruskal-Wallis test
Types of nonparametric statistics tests
Used when researchers are only able to make a few assumptions about distribution of scores in the underlying pop. Specifically, their use is suggested when nominal or ordinal data are involved or when interval or ratio data are not distributed normally (i.e., are skewed).
Non-parametric statistics test
A nonparametric statistical test used to determine whether two or more categorical or nominal variables are statistically independent.
EX: investigating rxp of whether to terminate counseling (yes/no) and the gender of the counselor (male/female)
Chi-square test
Nonparametric statistical test. Analogous to a parametric independent t-test, except uses ordinal data instead of interval or ratio data. Compares the ranks from two groups.
EX: Compare students Grades 9-12 w/education aspiration (diploma, 2/4-yr degree, grad) as DV
Mann-Whitney U-test
Nonparametric statistical test. Similar to Mann-Whitney U test, but used with samples smaller than 25 participants
Kolmogorov-Smirnov Z Procedure
Nonparametric statistical test analogous to an ANOVA. Extension of Mann-Whitney test. Used with 3 or more groups per independent variable & an ordinal-scaled dependent variable.
Kruskal-Wallis test
Nonparametric statistical test. Equivalent to a dependent t-test. Involves ranking the amount & direction of change for ea pair of scores.
EX: Comparing perceived level of competence before and after a training program
Wilcoxon’s signed-ranks test
A nonparametric statistical test similar to Wilcoxon’s signed-ranks test in that it is designed for repeated measures. It may be used with more than two comparison groups.
Friedman’s rank test
Graphical Representations of Experimental Designs:
- ___ = A, B, C, D
- ___ = (R)
- ___ = O
- ___ = X, Y, Z
- ___ = n/a
- Groups
- Random Assignment
- Observation
- Intervention
- control group (no intervention or observation)
A type of experimental design used when it is impossible or inappropriate to randomly assign participants to groups. Often used with nested data (e.g., classrooms, counseling groups) or naturally occurring groups (e.g., males, African Americans, adolescents). Two common types:
(a) nonequivalent groups pretest-posttest control (sometimes called comparison group designs), and
(b) time series design.
Quasi-experimental designs
Common type of quasi-experimental design.
Observations made at equal time intervals with same testing procedures. Characterized by repeatedly measuring before and after an intervention for one group, & uses a control group for comparison.
(Graphical Representation)
control group interrupted time series design
A: 0 -> 0 -> 0 -> X -> O -> O -> O
B: 0 -> 0 -> 0 -> n/a -> O -> O -> O
Common type of quasi-experimental design.
Counselor keeps groups intact; administers a pretest; administers treatment to only one group
(Graphical Representation)
nonequivalent groups pretest-posttest control group design
A: O -> X -> O
B: O -> n/a -> O
Common type of quasi-experimental design. Observations made at equal time intervals with same testing procedures. Characterized by REPEATEDLY measuring before & after intervention; can measure one group, or include control group for comparison.
time series design
Common type of quasi-experimental design.
Counselor keeps groups intact; administers pretest, administers treatment to 2+ groups, and then gives the groups a posttest.
(Graphical Representation)
nonequivalent groups pretest-posttest comparison group design
A: O -> X -> O
B: O -> Y -> O
C: O -> Z -> O
Common type of quasi-experimental design.
Characterized by repeatedly measuring before and after an intervention for one group.
(Graphical Representation)
one-group interrupted time series design
A: 0 -> 0 -> 0 -> X -> O -> O -> O
Aka randomized experimental designs. The gold standard; involve 2+ groups for comparison & random assignment. Common types
(a) randomized pretest-posttest control group design
(b) randomized pretest-posttest comparison group design,
(c) randomized posttest-only control group design,
(d) randomized posttest-only comparison group design, and
(f) Solomon four-group design
true experimental designs
