Unit 3: Validity and Control Flashcards
Define validity
an indication of accuracy – in terms of how closely the research conclusion corresponds to reality
internal validity
The extent to which the experiment demonstrates a cause-effect relationship. The degree to which the experimental manipulation alone caused the change in behaviour.
construct validity
the degree to which the study measures and manipulates the underlying concepts the researcher claims. Does the experiment study what it is supposed to?
external validity
the degree to which the results could be generalized to different subjects, settings, or times.
statistical conclusion validity
the degree to which the observed results are not due to chance only
several common categories of confounding variables, known as threats to internal validity
History Maturation Testing Instrumentation Regression effects Selection biases Mortality Subject variables
History
Specific events other than the independent variable that occur between the first and second test an influence the dependant variable.
Maturation
Internal changes to the subject (biological and psychological) that happen over time.
Testing effects
Any effects on the dependant variables as a result of repeated testing.
Instrumentation
Changes over time to the way the dependant variable is measured.
Regression effects
Subjects with extreme scores give scores closer to the average on subsequent testing.
Selection biases
When different selection procedures are used for placing subjects in the groups.
Mortality
Subjects from different groups dropping out at different rates. It is a threat when the ones who drop out are different than the ones who stay, thus biasing the subject group.
Subject variable
When characteristics can only be selected, not controlled.
Control
Any technique used to reduce threats to validity
Randomization
an unbiased assignment process that gives all individual subjects equal chance at being placed into any of the experimental groups.
Matching
A control technique used to ensure that groups are equated on one or more variable before the experiment.
Statistical control
A mathematical way of equating subjects on paper when they can’t be equated in fact.
Statistical control
A mathematical way of equating subjects on paper when they can’t be equated in fact.
Confounding
An error that occurs when the effects of more than one variable can not be separated -> confused interpretation of results.
Why is confounding an issue with subject variables?
When characteristics can only be selected, not controlled, there is a greater chance of other influences on the subjects and therefore on the results they give, which can not be separated from the variables being tested.
What are ways to determine construct validity?
- Manipulation check - part of the experimental design that checks to see whether the independent variable is changing in the intended way.
- Run another experiment to test other possible explanations.
2 threats to construct validity.
- When theory and method are too loosely connected.
- Ambiguous effects of independent variables - when subjects don’t behave naturally - good-subject tendency (giving the tester what you think they want), or evaluation apprehension (pretending to be the more socially acceptable way).
3 threats to external validity
- People - subjects are not representative of the population
- Time - tests done at another time yield different results (social factors change over time)
- Place - behaviour can change depending on the location.