Group Design Flashcards
Describe Experimental Designs: (5)
- Experimental Research – ACTIVE CONTROL of independent variable by the experimenter
-Can be group or single-subject - Individual or group design is based on how data are analyzed, not on how many participants are involved or how they are treated/tested
- Group: Individuals’ data are combined for analysis
Single-subject: Each individual’s data are analyzed separately - Some group studies may provide individual data as a follow-up to group results
Experimental Group Designs is highly _________________ and has the highest ___________ _______________ if well designed.
Highly controlled: highest internal validity if well designed
What makes the Experimental Group design Highly controlled and highest internal validity? (3)
- Manipulation of independent variable
- Appropriate control(comparison) group/condition
- Random assignment to group/condition
What makes Quasi-experimental design weaker than Experimental design?
- Sometimes impossible to meet stringent experimental requirements
- Weaker design than true experimental
- Often treated like experimental
- Not all questions allow experimental (e.g., comparing TD and TBI – no random assignment)
What are two types of assignments in Quasi-Experiments? (Consequences and Examples)
A) no control group/condition
Consequence: no comparison group
e.g., simple pretest-treat-posttest; all participants experience IV
B) no random assignment
Consequence: possibly non-equivalent groups
Examples
Naturally occurring
E.g., treatment 1 in school ‘A’, treatment 2 in school ‘B’
People on waiting list
History control group – file review of patients before offered experimental treatment
Those who declined treatment as control group
What is the internal Validity in Quasi-Experimental designs?
Internal validity & confidence in results is reduced
Less faith in attributing the cause of the change the treatment
How can we increase validity in quasi-experimental studies? (3)
- Describe subjects carefully
- Test for equivalence (but can’t test for everything)
- Match groups on critical parameters
-Choose control participants so that they match experimental participants on parameters, e.g., MA, SES
What is Between Subjects design?
- Separate groups of independent participants who are sampled only once
- If multiple measures given, all comparisons between groups
Compare Grp 1 vs Grp 2 on measure 1
Compare Grp 1 vs Grp 2 on measure 2 - Less common in treatment studies
- When use ‘gain scores’
What is the difference between N and n in group designs?
‘N’ – total number of participants in study
‘n’ – number of participants in each group
What is the difference between N and n in group designs?
‘N’ – total number of participants in study
‘n’ – number of participants in each group
What are the assumptions in Experimental Group designs?
- Variability in data is intrinsic to participants
- Large groups and random assignment balances this noise/error in the study between the control & experimental group
- When balanced, assume a difference between groups is the result of the independent variable
- Randomization gives you balanced groups
- Inferential statistical analysis on grouped data allows the determination of ‘significant’ difference
- Can generalize to population (assumes sample represents population)
What is Within Subject/Related Samples/ Repeated Measures?
- Participants sampled on same measure more than once.
- One Grp compared at Time 1 and Time 2
- Single-group pretest-posttest design & Time series design
OR
- Participants are related (e.g., siblings, classmates)
-1st born are compared to 2nd born
OR - Participants sampled on multiple conditions/measures and comparisons made within the group
-One Grp condition/measure1 compared to condition/measure 2
N = number of participants
What is a concern in Within Subject/Related Samples/ Repeated Measures?
Order or carry-over effects,
e.g., Participants perform a task better in later conditions because they have had a chance to practice it.
What is the most common group design in Experimental designs?
Mixed (Between +Within Subjects)
What are the characteristics of Mixed group design?
- Contains both between and within-subject variables.
- Standard treatment study
Treatment group vs control group (between)
Tested Pretest & Posttest therapy (within)
Here is an example of Comparing hearing aid 1 vs 2 (Between Within and Mixed)
Between
N = 20 in 2 groups; n = 10
Grp 1 = HA-1; Grp 2 HA-2
IV: type of aid; DV: speech perception score
Within
N = 20; n = 20
Each participant wears HA-1 1 wk, HA-2 1 wk
IV: type of aid; DV: speech perception score
To control order effect, better if ½ do HA-1 1st & ½ do HA-2 1st
Mixed
10 mild HL, 10 moderate HL, 10 severe HL – N = 30; n = 10
Each participant wears HA-1 1 wk, HA-2 1 wk
IV between: hearing level; IV within: type of aid; DV: speech perception score
To control order effect, better if ½ do HA-1 1st & ½ do HA-2 1st in each group
Here is an example of Comparing hearing aid 1 vs 2 (Between Within and Mixed)
Between
N = 20 in 2 groups; n = 10
Grp 1 = HA-1; Grp 2 HA-2
IV: type of aid; DV: speech perception score
Within
N = 20; n = 20
Each participant wears HA-1 1 wk, HA-2 1 wk
IV: type of aid; DV: speech perception score
To control order effect, better if ½ do HA-1 1st & ½ do HA-2 1st
Mixed
10 mild HL, 10 moderate HL, 10 severe HL – N = 30; n = 10
Each participant wears HA-1 1 wk, HA-2 1 wk
IV between: hearing level; IV within: type of aid; DV: speech perception score
To control order effect, better if ½ do HA-1 1st & ½ do HA-2 1st in each group
What is the difference between longitudinal and cross-sectional?
Longitudinal: observational research technique involves studying the same group of individuals over an extended period. Variables that are not related to various background variables
Cross- Sectional: Type of research design in which you collect data from many different individuals at a single point in time. In cross-sectional research, you observe variables without influencing them
What is the difference between a Prospective and Retrospective study design?
Prospective: individuals are followed over time and data about them is collected as their characteristics or circumstances change
Retrospective: Individuals are sampled and information is collected about their past
Explain Degrees of Freedom:
When computing any statistical test, Degrees of Freedom (df) need to be determine, the size and number of groups being compared
The number of independent pieces of information used to calculate a statistic.
the extent to which components in a design are free to vary
What are parametric statistics? (5)
- Normal distribution
- Homogeneity of variance (groups have about the same amount of variance)
- Interval or ratio data
- Large ‘N’
- Equal ‘n’
If you violate too many parametric statistics you should use:
Nonparametric
With which type of data do you often see parametric analyses?
Often see parametric analyses used with ordinal data – e.g., from rating scale
Why are df necessary?
Necessary to determine critical value for test statistic to reach alpha level (i.e., p value associated with statistic).
What is df influenced by?
Influenced by number of participants, their scores and the number of independent (i.e. grouping) variables
As df __________________ (larger N, fewer groups give larger df) critical value of statistic (t, F etc.) _____________so easier to reach significance
As df increases (larger N, fewer groups give larger df) critical value of statistic (t, F etc.) decreases so easier to reach significance
What is the meaning of univariate?
Only 1 dependent variable
What is Multivariate?
More than 1 dependent variable
e.g., receptive & expressive language tests
e.g., assess HA via speech discrimination & client satisfaction
What is a Factorial design?
- More than 1 independent variable
groups (between)
e.g., male & female AND genetic & noise exposure HL
conditions (within)
e.g., pre- & post-Rx AND with- & without speech bulb
Both groups and conditions (mixed)
e.g., Rx1 / Rx2 (between) AND pre-Rx/post-Rx (within)
What is Multivariate Factorial Design?
- More than 1 dependent & more than 1 independent
e.g., examples above when measure more than one outcome (dependent variable)
In this research question: Are cats more intelligent than dogs?
IV, DV and the number of groups, univariate factorial or multivariate factorial design?
In this research question: Are cats more intelligent than dogs or crows?
IV, DV and the number of groups, univariate factorial or multivariate factorial design?
In this research question: Are cats more intelligent and friendlier than dogs?
IV, DV and the number of groups, univariate factorial or multivariate factorial design?
In this research question: Is intelligence in cats and dogs affected by indoor vs outside?
IV, DV and the number of groups, univariate factorial or multivariate factorial design?
In this research question: Is intelligence & friendliness in cats and dogs affected by indoor vs outside
IV, DV and the number of groups, univariate factorial or multivariate factorial design?
In design with 1 IV fill in the blanks:
Between Subjects designs
Two:
Two:
More than two:
More than two:
Within Subject designs
Two:
Two:
More than two:
Between Subjects designs
Two groups parametric
Two groups non-parametric
More than two groups parametric
More than two group nonparametric
Within Subject designs
Two groups parametric
Two groups non-parametric
More than two groups parametric
What are the assumptions for the parametric independent t-test?
Assumptions for parametric
Data?
Distribution?
Variance?
N?
n?
Other Assumptions
Sample drawn?
Dependency of groups?
Assumptions for parametric
Continuous data - interval or ratio
Normal distribution
Homogeneous variance
Large ‘n’
Equal ‘n’
Other Assumptions
Sample drawn randomly
Independent groups
What is a t-test?
Statistical test that is used to compare the means of two groups.
In this question: Does a parent information night lead to fewer missed appointments?
IV & DV?
Split groups to create Experimental design.
½ parents attend information night, ½ do not, randomly assign (experimental)
Track the number of missed/cancelled appointments over the year
In this question: Does a program on hearing aids for nursing home staff increase the residents’ wearing of hearing aids?
IV & DV?
Split groups to create quasi-experimental design.
Staff of ½ nursing homes get training, ½ do not, randomly assign nursing homes (quasi-experimental)
Number of days clients wear hearing aids
What is a t-statistic based upon?
t-statistic calculation is based on the relationship between the group means and their variances
What would be the formula of df?
Degrees of freedom:
df = (n1 – 1) + (n2 – 1) = N – 2
K groups = K degrees of freedom
What are the steps in testing for the hypothesis in t-test?
- Test the null hypothesis: the 2 means do not differ significantly H0: μ1 = μ2
- If the t-statistic is larger than a critical value at a specified level of significance (e.g., α = .05), then the null is rejected and the alternative accepted
H1: μ1 ≠ μ2 (2-tailed)
H1: μ1 > or < μ2 (1-tailed)
Effect size if significant – e.g., Cohen’s d, ɳ2 - Show results:
(t(df) = x.xx, p = 0.xx)
In this non-parametric analysis: Are people with vocal nodules rated as more hoarse (on a 5-point scale) than people with vocal polyps?
What are IV and DV, number of groups?
In Non parametric Analyses: describe the Mann-Whitney U test in 2 groups and 1 DV:
Mann-Whitney U test
Ordinal data
U statistic
In Non parametric Analyses: describe the Wilcoxon rank sum test in 2 groups and 1 DV:
Wilcoxon rank sum
Ordinal data
Z statistic
In Nonparametric Analyses: describe the Chi-Square test for Two Independent Samples test in 2 groups and 1 DV:
Chi-Square test for Two Independent Samples
Nominal data (e.g., % Ss have cancer living in 2 different communities)
Chi-square statistic
What is ANOVA? (3)
Also called one-way ANOVA
1 independent variable with three or more levels
Used to test more than 2 mean differences
What tests are more efficient than multiple t-tests?
ANOVAs
Alpha level is partitioned within the design and therefore provides a more powerful test of differences
What are the Assumptions of One-Way ANOVA (between subjects)?
Assumptions for parametric
Data and scale?
distribution?
variance?
‘n’?
‘n’?
Other assumptions
Sample drawn?
Dependency groups?
Assumptions for parametric
Continuous data - interval or ratio
Normal distribution
Homogeneous variance
Large ‘n’
Equal ‘n’
Other assumptions
Sample drawn randomly
Independent groups
In this research question: Does type of hearing loss impact speech discrimination skills?
Identify IV and DV
Split your groups to create an ANOVA between subject model)
Individuals with a moderate conductive hearing loss, a moderate sensori-neural or moderate mixed loss (3 groups)
Look at performance on the SPIN (Speech in Noise) test
In this research question: Do children with DLD, DS, Fragile X, and TD, matched on mental age, differ in their vocabulary skills?
IV & DV?
Split Groups to create a One-Way ANOVA
4 groups of children all with MA = 2 years
Measure their receptive vocab skills on PPVT-5
In a One-Way ANOVA
What are the 3 types of df?
Total df
(N-1) for a between-subject design
Group df
(Number of groups – 1)
Error df
(Total df – group df)
What statistic do we use to report findings on a One-Way ANOVA?
F(grp df, error df) = xx.x; p = 0.xx)
then
Compared to critical level to determine if significant
What are the 3 first steps in a One-Way ANOVA (between subjects)
- Statistical hypotheses
One-way ANOVA
H0: no difference (i.e., the observed differences between a set of means are no greater than expected by chance alone)
H0: μ1 = μ2 = μ3 - If reject, there is a statistically significant difference AMONG the groups
- Don’t know where the difference(s) is/are
1-2, 2-3, 1-3 so Post-hoc testing
What are Post-hoc testing?
Used to follow up significant F to determine which groups differ significantly
t-tests: e.g. 1 vs 2, 2 vs 3, 1 vs 3
What are the most to least powerful Post-hoc Multiple-comparison controls?
Less Planned orthogonal contrasts
Newman-Keuls
Tukey HSD
Bonferroni
Most Scheffé
In this research question: Does degree of stuttering behaviour (mild/moderate/severe/profound) affect judgment of intelligence (below average, low average, average, high average, above average) by naïve listeners?
IV & DV? number of groups?
Which tests to use when having > 2 groups with 1 DV? (2)
Kruskal-Wallis One-Way Analysis of Variance
Chi-Square Test for K Independent Samples
Describe Kruskal-Wallis and Chi-square test for >2 groups and 1 DV:
> 2 groups
Kruskal-Wallis One-Way Analysis of Variance
Type of data?
Statistic?
Post-hocs test?
Chi-Square Test for K Independent Samples
Type of data?
Statistic?
When more than 2 groups:
> 2 groups
Kruskal-Wallis One-Way Analysis of Variance
Ordinal data
Like 1-way ANOVA
H statistic
Mann-Whitney for post-hocs
Chi-Square Test for K Independent Samples
Nominal data
Chi2 Statistic
What tests to use when Within subject/Dependent Sample Designs and 1 Dependent variable, 1 Independent variable? (3)
Paired t-test
1-way ANOVA
Non-parametric
What tests to use when Within subject/Dependent Sample Designs and 1 Dependent variable, 1 Independent variable? (3)
Paired t-test
1-way ANOVA
Non-parametric
What are paired t-tests?
A test used when comparing 2 dependent groups or
dependent samples, repeated samples and paired samples.
What are the Assumptions for PAired (correlated t-test)?
Assumptions for parametric
Type of data and scale?
Distribution?
Variance?
‘n’?
Other Assumptions
Dependency on groups?
Sample drawn?
Assumptions for parametric
Continuous data - interval or ratio
Normal distribution
Homogeneous variance
Large ‘n’
Equal ‘n’
Other Assumptions
Dependent groups
Sample drawn randomly
In this research question: Do first-born children have larger vocabularies than 2nd born children?
Create a Paired t-test and identify IV and DV
IV:
DV:
Groups: Test 1st born at 24 months
Test their siblings at 24 months
Tested on the CDI a vocabulary checklist completed by parents – number of words reported
Correlated t-test as siblings are not independent
In this research question: Does the SuperDuper ‘ed’ program result in immediate improvement in children’s past tense marking?
Create a Paired t-test and identify IV and DV
IV:
DV:
Groups:
‘ed’ test – run program – ‘ed’ test
Fill in the blank related to (paired t-tests)
Statistic ____
Degrees of Freedom _____
Compare ____ to _________ level
Effect size if significant – e.g., Cohen’s d, ɳ2
(t(df) = x.xx, p = 0.xx)
Statistic t
Degrees of Freedom
N-1
Compare p to alpha level
Effect size if significant – e.g., Cohen’s d, ɳ2
(t(df) = x.xx, p = 0.xx)
What are the nonparametric: paired tests we use in 2 groups, 1 IV?
Wilcoxon (Matched Pairs) Signed Ranks Test
McNamara’s Test
Describe Wilcoxon and McNamara tests:
Wilcoxon (Matched Pairs) Signed Ranks Test
Ordinal data
2 related samples
McNamara’s Test
Nominal data
2 related samples
What are statistical procedures frequently used with nominal data?
What are statistical procedures frequently used with ordinal data?
What are statistical procedures frequently used with ratio data?
