Terminology and Statistics Flashcards
Between groups
looking at differences BETWEEN different groups
ex. Group 1 vs. Group 2
Within groups
-repeated measures
looking at differences WITHIN (same person over time)
ex. one participant: T1 vs T2 vs T3
Analysis of Variance (ANOVA)
- One- way = 1 IV
- Two-way = 2 IV
- Factorial = 2+ IV
F statistic
- Ratio of the variation explained by the model and the variation explained by unsystematic factors
- If an f statistic is 1 or greater, chances are it will be significant
The OVAs
ANOVA- analysis of variance
ANCOVA- analysis of covariance (1+ covariate)
MANOVA- multivariate analysis of variance ( 2+ DVs)
MANCOVA- multivariate analysis of variance (2+ DVs, 1+ covariate)
-Mixed- model design ANOVA
Mixed model design ANOVA
- gets its name because there are two types of variables involved, that is at least one between-subjects variable and at least one within-subjects variable.
- The mixed-design ANOVA model (also known as Split-plot ANOVA (SPANOVA)) tests for mean differences between two or more independent groups whilst subjecting participants to repeated measures. Thus, there is at least one between-subjects variable and at least one within-subjects variable.
Example of a mixed model design ANOVA
- For example, are there any differences amongst the heights of males and females at age 10 and age 20 years?
- Gender (male or female) is the between-subjects variable
- Age (10 or 20 years) is the within-subjects variable
- Of interest are the main effects for Gender and Age, and the Gender-Age interaction effect.
- This could be described as a 2 x (2) mixed-design ANOVA
Main effect
with example
- the overall effect of one independent variable.
- EXAMPLE: In an experiment in which both the type of psychotherapy (cognitive vs. behavioral) and the duration of psychotherapy (short vs. long) are independent variables, there is one main effect of type and another main effect of duration. The main effect of type is the difference between the average score for the cognitive group and the average score for the behavioral group … ignoring duration. That is, short-duration subjects and long-duration subjects are combined together in computing these averages. The main effect of duration is the difference between the average score for the short-duration group and the average score for the long-duration group … this time ignoring type. Cognitive-therapy subjects and behavioral-therapy subjects are combined together in computing these averages.
Interaction effects
with example
-In order to find an interaction, you must have a factorial design, in which the two (or more) independent variables are “crossed.” a special kind of effect that can be observed in factorial experiments. You have an interaction whenever the effect of one independent variable depends on the level of the other. This is actually a fairly easy idea.
Here are some examples. The combined effect of two or more variables, as demonstrated in a factorial design; interactions signify that the effect of one variable (e.g., sex of the subject) depends of the level of another variable (e.g., age)
- If cognitive psychotherapy is better than behavioral psychotherapy when the therapy is short but not when the therapy is long, then there is an interaction between type and duration of therapy.
- If the negative effect of noise level on concentration is greater for introverts than for extroverts, then there is an interaction between these two independent variables.
- If the boost in intelligence judgments due to smiling is greater for male stimulus persons than for female stimulus persons, then there is an interaction between smiling and sex.
- If drawing a smiley face on checks increases tips for female servers but not for male servers, then there is an interaction between drawing smiley faces (or not) and sex of the server.
crossover interaction
-the effect of one independent variable is not only different across levels of the second independent variable, it actually reverses. Imagine, for example, that introverts’ concentration levels started high and then dropped as the noise level increased, but extroverts’ concentration level started low and then increased as the noise level increased. This would be a crossover interaction.
Buzz words:
predictors =
relationship =
adjusting =
post-hoc =
regression
relationship
adjusting
three groups
Discriminant Function Analysis (DFA)
- quantitative variable to predict group membership (Categorical)
- reverse MANOVA
homoscedasticity (homogeneity of variance)
- Condition in which all the variable in a sequence have the same finite, or limited, variance
- When homogeneity of variance is determined to hold true for a statistical model, a simpler statistical or computational approach to analyzing data may be used due to a low level of uncertainty in the data
Two types of covariates:
mediator
moderator
moderator
a variable that INFLUENCES the strength of a relationship between two other variables
example: in arguments/debates, the two parties agree from the beginning to have a conversation. The moderator is simply there to help manage the strength of the two parties’ arguments. i.e., this variable has an effect for sure, but the relationship will be there one way or the other
mediator
a variable that EXPLAINS the relationships between two other variables
example:
- in disputes, mediators are usually called in when two parties are unable/willing to communicate. It all hinges on the mediator to help solve the problem. i.e., if the mediator is gone, the relationship is nonexistent!
Example of mediator vs moderator
-What’s the relationship between social class (SES) and frequency of breast self-exams (BSE)? Age might be a moderator variable, in that the relationship between SES and BSE could be stronger for older women and less strong or nonexistent for younger women. Education might be a mediator variable in that it explains why there is relation between SES and BSE. When you remove the effect of education, the relationship between SES and BSE disappears.
confounding variable
a variable that is influencing both the DV and IV and is not accounted for. Failure to account for confounding variables can lead to spurious relationship
Example: ice cream sales impact drowning deaths. This is a spurious relationship because it fails to account for season
covariate
a variable that has a relation with one or both of the IV and DVs, but does not appreciably change the relation between an IV and DV when included in a statistical analysis. Covariates are generally not of theoretical interest, but are often included in a model to explain additional variability in a DV
regression
used in prediction and forecasting
Types: linear, bivariate, logistic, quadratic, exponential, logarithmic, power, etc.)
bivariate regression
degree of relationship between two quantitative variables
logistic regression
degree of relationship between categorical variables
linear regression
when both the predictor and response variables are continuous and linearly related, so the response will increase or decrease at a constant ratio to the predictor. There can be more than one predictor
example of linear regression
As the number of farms has decreased in the U.S., the average size of the remaining farms has grown larger. Based on data (year and the corresponding average acreage/farm) predict the average acreage in 2000 and 2010 and determine which function gives the most realistic predictions
logistic regression
- the predictor is continuous, but the response is categorical or dichotomous. The logistic regression provides you with the probability of an event occurring. For every one unit of increase in the predictor, the probability of an event occurring increases or decreases.
- The information of interest is the probability of an individual falling into one of the two populations given the scores of the IVs
- Originally used in medical research where the DV was the presence or absence of a disease and the IV’s were risk factors
example of logistic regression
Suppose that we are interested in the factors that influence whether a political candidate wins an election. The outcome (response) variable is binary (0/1): win or lose. The predictor variables of interest are the amount of money spent on the campaign, the amount of time spend campaigning negatively, and whether or not the candidate is an incumbent.
Statistical conclusion validity
the extent to which a relation between independent and dependent variables can be shown, based on quantitative and statistical considerations of the investigation. Although Cook and Campbell seemed to base this on the preponderance on inferential statistic-based research in psychology, Kazdin notes that not all psychological research is based on inferential statistics (e.g., qualitative research and single-case experimental designs). He therefore recommends the term, data evaluation validity
Threats to statistical conclusion validity
Mondays: Law and order: SVU
- Multiple comparisons and errors rates
- low statistical power
- subject heterogeneity
- variability in the procedures
- unreliability of the measures
Multiple comparisons and errors rates
Threat to statistical conclusion validity
-When there are multiple comparisons, alpha increases beyond point .05
E.g., if you gave different measures to clients, clinicians, and client family members, on a variety of areas, the likelihood that there will be differences across the ratings increase by chance alone
low statistical power
Threat to statistical conclusion validity
most common threat
the study is not designed to detect a difference when there very well may be one
subject heterogeneity
Threat to statistical conclusion validity
the greater the diversity of the sample, the more difficulty it will be to detect a difference between groups
variability in the procedures
Threat to statistical conclusion validity
While not all variability can be controlled for, failing to minimize variability will impact effect size, biasing results
Pg. 72, “In terms of our formula for effect size, the differences between groups will be divided by a measure of variability; this measure will be larger when there is more, rather than less, uncontrolled variation. The larger the variability, the lower the effect size evident for a given difference between groups.
unreliability of the measures
Threat to statistical conclusion validity
self explanatory
Construct validity
Kazdin, 2003
In the context of experimental design, this refers to a type of experimental validity that pertains to the interpretation or basis of the effect that was demonstrated in an experiment. (in the context of a psychological assessment the term refers to extent to which a measure has been shown to assess the construct [e.g., intelligence] of interest)
Threats to construct validity
CASE
- Cues of experimental situation
- Attention and contact with clients
- Single operations and narrow stimulus sampling
- Experimenter expectancies
Cues of the experimental situation
Threats to construct validity
-Those seemingly ancillary factors associated with the intervention that may contribute to the results
-These cues have been referred to as demand characteristics (Orne, 1962)
E.g., information conveyed to prospective participants prior to their arrival (rumors about experiment, information provided during recruitment), instructions, procedures, and any other features of the experiment
Attention and contact with clients
Threats to construct validity
Attention to the clients, rather than the intervention itself, might be responsible for change
E.g., In placebo trials, for instance, the intervention of a drug may clearly work (internal validity); but that does not mean that the best conclusion is that it was the drug itself that yielded the results, rather than other aspects of the intervention.
Single operations and narrow stimulus sampling
Threats to construct validity
An experimental manipulation or intervention that includes features that the investigator considers to be irrelevant to the study, but these features later introduce ambiguity in interpreting the findings.
E.g., Psychodynamic therapy as measured by veteran analysts vs. CBT administered by social workers in their first year. Were the treatments being studied, or was this a study measuring the skills of the first group vs. the skills of the second?
Experimenter expectancies
Threats to construct validity
-The expectancies, beliefs, and desires of the experimenter (the one administering treatment) for the results influences how participants perform
Internal validity
Kazdin, 2003
– the extent to which the experimental manipulation or intervention, rather than extraneous influences, can account for the results, changes, or group differences.
-the “approximate validity with which we infer that a relationship between two variables is causal” (Cook and Campbell, 1979. P.37).
• A good synonym for the term internal validity is causal validity because that is what internal validity is all about.
• If you can show that you have high internal validity (i.e., high causal validity) then you can conclude that you have strong evidence of causality; however, if you have low internal validity then you must conclude that you have little or no evidence of causality
Threats to internal validity
DIRT CHASMS
- Diffusion or imitation of treatment
- instrumentation
- statistical regression
- testing
- combination of selection and other threats
- history
- attrition
- selection bias-
- maturation
- special treatment or reactions of controls
Diffusion or imitation of treatment
Threats to internal validity
experimental conditions are unintentionally similar to one another (e.g., behavioral therapy vs. eclectic, that unintentionally includes behavioral methods)
Instrumentation
Threats to internal validity
the measuring instrument or measurement procedure can change over time (e.g., WAIS-R vs. WAIS-IV)
Statistical regression
Threats to internal validity
statistically, individuals initially testing with extreme scores tend to revert towards the mean during the second testing
Testing
Threats to internal validity
exposure to the test previously (as in pre-test/post-test) might account for a change in performance, not the experimental manipulation
Combination of selection and other threats
Threats to internal validity
when there are different internal validity threats for different experimental groups (e.g., group 1 = experienced the world trade center attacks, group 2 = did not experience this historical event)
History
Threats to internal validity
events both internal and external to the experiment not accounted for by the IV (and otherwise not controlled for); ex. what’s going on in the news, weather, that may account for results
Attrition
Threats to internal validity
loss of subjects over time (drop-out)
Selection bias
Threats to internal validity
systematic differences between experimental groups prior to the experiment
Maturation
Threats to internal validity
changes over time (e.g. participants getting older, wiser, bored, etc.)
Special Treatment of reactions of controls
Threats to internal validity
The control group is unintentionally given an incentive or condition that was not controlled for (e.g., higher compensation), that could be affecting results
External validity
the extent to which the results can be generalized or extended to persons, settings, times, measures, and characteristic other than those in this particular experimental arrangement.
Threats to external validity
MRS. TReNTS (NO E)
- Multiple treatment interface
- reactivity of experimental arrangements
- sample characteristics
- test sensitization
- reactivity of assessment
- novelty effects
- timing of measurement
- stimulus characteristics
Multiple- treatment interface
threats to external validity
to what extent does the interaction of receiving multiple experimental conditions (i.e. such as in a within subjects design) affect the results?
Reactivity of experimental arrangements
Threats to external validity
-How does the fact that one knows they are in an experiment affect the results
Pg 95 of Kazdin
Good – find out hypothesis (you think), and try to collaborate with it
Negativistic – find out hypothesis (you thin),and try to contradict it
Faithful – comply with all instructions
Apprehensive – try to portray yourself favorably
sample characteristics
threats to external validity
do the characteristics of the sample limit generalizability?
Test sensitization
threats to external validity
Giving a pretest might sensitize a person to the construct you are measuring (e.g., giving a Beck Depression Inventory before the participant watches a sad film might alert them to depressing stimuli, confounding results)
Reactivity of assessment
threats to external validity
this is participant knowledge that they are being assessed in the experiment
Novelty effects
threats to external validity
interaction with unfamiliar stimuli might be more responsible for results (e.g. imagine an experiment where someone had to remember a neon-pink dinosaur versus a picture of a chair)
Timing of measurement
threats to external validity
Does the timing for assessing experimental group bias results (e.g. evaluating therapy effectiveness a year from termination as opposed to 6 months)
Stimulus characteristics
Threats to external validity
to what extent is the setting affecting the results (e.g. for treatment studies, controlled clinical trials do mimic real-world clinical practice settings)
