FINDING AND FIXING FLAWS Flashcards
What would the only variance be in a perfect experiment?
Attributable to our manipulation - treatment variance
Whats extraneous variable ?
Variables that might affect our depdent variable but aren’t ones that we’re interested in.
For example child’s sex will affect their height but it’s not something we are interested in.
What kind of variance does extraneous variables look at?
Source of random variance or source of confound variance
Whats error variance?
Random error, this would be expected to affect all conditions equally but may make it harder to see tur effects
Whats cofound variance?
This will affect conditions differently
This is systematic as w can predict its effects
Describe random variance?
If we measure the heights of 100 adolescents we will expect that the height of each participant will be different to everyone else
- this would be expected by chance and will be for a variety of reasons
What happens when the random variance is large?
Make it difficult to see overall differences between groups
In psychology how can we reduce random variance?
Using homogenous groups
So in the study we only focus on females
What can confounding variables lead us to?
• Wrongly identify the causal component of the independent variable
• Assume the intended independent variable has an effect when it doesn’t
• Assume an effect doesn’t occur when it does
What does addressing confounds include?
• Control conditions
• Statistical controls
• Improved experimental design
• Random assignment of participants
Explain threats to validity?
An influence on our indepdent/ depdent variables that might provide an alternative explaination of our effect or limit the generality
Explain type 1 error?
When you reject the null hypothesis but the null hypothesis is true and its a false positive
Explain type 2 error?
When they dont reject the null hypothesis but null hypothesis is false
False negative
Define power in stastical validity?
The ability to see an effect of the indepdent variable on the depdent variable if it actually there
Define effect size in statistical validity?
The strength of the relationship between he indepdent and depdent variable
State the threats to statistical validity?
• Erroneous choice of statistics
• If the wrong test is chosen, the results cannot be trusted
• Limited Power
• The effect may not be observed because of too few subjects
• Inaccurate Effect Size estimation
• The level of relationship between the Independent and Dependent Variable is
not correct
Whats internal validity?
Did the manipulation of the iv cause the observed changes in the DV?
What happens when theres no internal validity?
The effects of the independent variables are confounded
- when the effects of two variables cannot be separated or interpreted
What are the threats to internal validity?
• History – Events outside the lab that may bias your conclusions.
• Selection (biased sampling) – Picking subjects in a way that is not
random
• Attrition (or Mortality) – Some subjects drop out
• Maturation – Effect of time between measurements
• Effects of Repeat Testing – Learning the test and get better at it
• Regression Effect – Extreme scores tend to move toward the middle
on a second test.
How to address the threats to internal validity?
- ensure participants in each group are treated as similar as possible
Maturation effect
Attrition effect
• Gather data rapidly. Or, make sure that all groups/conditions are
being gathered simultaneously
History effect
• Random Assignment
Selection effect
• Counterbalancing (for within subjects designs)
• Statistical control
Define construct validity?
Does the experiment manipulate/ measure the concept in question
Crucial in psychology because we deal with psychological constructs
What construct validity apply to?
Manipulation of the indepdent variable or your measurement of the dependent variable
The measurement tests in construct validity?
• Measures the idea it’s supposed to, and nothing else
• Face Validity
• Test seems superficially valid
• Content Validity
• Tests samples the range of behavior represented by the theoretical
concept, not part
• Criterion Validity
• Test correlates to other tests of the same idea
Threats to construct validity?
• Loose connection between theory and method.
• Poor operational definitions
• Poor measurement techniques
• Confounds
