Slides 3 Flashcards
construct validity
the extent to which a measure can assess the construct of interest
how to test construct validity
correlations between related constructs
what does construct validity mean in an experimental context
what is actually driving the effects we are seeing in the experiment (assuming we have internal validity)
major threats to construct validity
placebo and participant expectancy - especially if the participant believes the treatment will be good
what is an active placebo
a pacebo that produces some of the somatic effects, mimicking the side effects of the actual treatment, but with no activity ingredient to remedy anything
single-operational/narrow stimulus sampling
- what is it
- what is it a threat to
it is when there is another factor that participants may be responding to other than the treatment. For example a therapist if the therapist is there for all of the experimental participants it could be that the therapist is really good rather than if the type of therapy worked.
- threat to construct validity
what are demand characteristics
cues of the situation associated with the study that seem incidental but may account for the results
- often happens during the informed consent process
statistical conclusion validity:
the extent to which the analysis preformed enables one to draw correct inferences about the phenomena of interest
state some threats to statistical conclusion validity
low power (small N, small effect size)
variability in procedures –> increases error in measurments
subject heterogeneity
unreliable measures
multiple comparisons
definition of concept: alpha
the probability of rejecting a hypothesis when that hypothesis is true
also called type 1 error
definition of concept: beta
the probability of accepting a hypothesis when it is false
also referred to as type II error
definition of concept: power
the likelihood of finding differences between conditions when in fact, the conditions are truly difference
also defined as 1-beta
effect size
a way of expressing the magnitude of the difference between conditions in terms of a common metric across measures and studies
what happens to the effect size when there are more methological problems in the study
the effect size gets smaller (even if in nature, the effect size is large)
effect size can be impact in two general ways:
1) increasing the difference between means
2) reducing the standard deviation by controlling for methological factors that increase variance
threats to data-evaluation validity
several facets of the results and statistical evaluation that can obscure interpretation of the experiment
kinds of threats to data evaluation validity
variability in procedures
unreliability of measurement
restricted range of measures - can’t show all group differences
multiple comparisons and error rates - when more stat tests are run, the likelihood of finding a “chance” finding is increased
how to increase range of individuals from whom volunteering is sought
target individuals instead of general volunteer inquiries
helps entice individuals to volunteer from the age groups you want
______ : who remains in the study after dropouts
attrition
ways to combat attrition if you are planning a study
backload money (frowned on) - more ethically acceptable is a completion bonus
target individuals who are less likely to back out
- not good for external validity
describe the good subject
attempts to provide responses/behaviours that will corroborate their perceptions of investigator’s hypothesis
explain the negativistic subject
attempts to provide response that will refute the investigates hypothesis, or provide no use
the faithful subject
attempts to follow experimental protocal carefully and avoid acting on beliefs about studys purpose
explain the apprehensive subject
when participant is concerned that their performance will be used to evaluate abilities, personality characteristics
- motivated to present sefl in most desirable light
best defences against the different types of subjects
make sure they know enough to make an informed choice, but not enough to know the hypothesis
-to do this you could include more details than necessary/ in study so they don’t know what is involved I the study
what is something that is often overlooked when collecting data even on huge research projects
data errors
what is something like the file drawer effect that leaves information collected during a study out
selective reporting of results
file drawer problem - why does it happen?
bias towards publishing significant effects
selection biases
convenience samples and volunteers - over representation of undergrad psychology students - the majority of which are female -
- may select certain types of convenience or volunteer samples to answer certain questions
- highly educated, middle class, high IQ, sociable ppl with females , young, and jewish ppl volunteering the most
what is reactivity
Reactivity of experimental arrangements
Awareness of being in a study may affect behaviour or elicit certain reactions
Usually increase productivity, performance
May not generalize to situations where you don’t feel like you’re being watched or you’re being monitored
