Selecting Measures and Non-Experimental Methods I: Observational and Survey Research Flashcards
every measure we obtain consists of:
“true score” and error
error is due to:
- bias (a systematic deviation that is the result of confounds)
- random error (a result of nuisance)
what are three sources of measurement error?
- experimenter
- participant
- observer/scorer
experimenter error examples?
random error, and bias error (experimenter characteristics and expectancies)
solutions for experimenter errors?
standardize testing conditions, standardize appearance of experimenter/replicate experiment with different experimenters, standardize coding schemes/automated recording equipment/single blind research
examples of participant error
carelessness and distraction (contributes to nuisance) and participant bias
solutions for participant errors
set clear task instructions with emphasis on accuracy, include a manipulation check
what are some causes for participant bias
Demand characteristics and good participant effect together can cause a “pact of ignorance” - invalidates results
demand characteristics
features of an experimenter that seem to inadvertently cause participants to act in a particular way
good participant effect
tendency for participants to behave as they perceive the researcher wants them to behave
how to control for demand characteristics
conduct double-blind research (removes confounds, but not nuisance), can use deception but this could inadvertently cause demand characteristics
response set
when the context affects the way a participant responds, can be a factor of the experimental setting or the questions that are asked (social desirability could influence answers)
response set contributes to:
response bias (participant bias)
to control for yea/nay-sayers:
include both agree and disagree terms with switched implications, randomize question presentation (reverse-coding), care review of response set, use of pilot tests
observer error is only present in:
behavioural studies
examples of observer error
random observer error, observer/scorer bias-confirmatory bias
how to control for observer error:
eliminate human observer (muse mechanical measures), limit observer subjectivity (standardized coding schemes), make observer “blind”
construct validity
the extent to which the manipulation or measure actually represents the claimed construct
- establishing reliability (criteria for validity)
test-retest reliability, inter-rater reliability, internal consistency (research finding must be repeatable and consistent)
what is internal consistency
extent to which responses to items that propose to measure the same unidimensional construct are similar, test by Split-half correlation, Cronbach’s Alpha, improved by adding items/questions
- content validity (criteria for validity)
extent to which a measure covers all aspects of a construct
- convergent validity (criteria for validity)
extent to which a measure correlates with other indicators of the same construct
- discriminant validity (criteria for validity)
extent to which your measure is distinguishable from other constructs (both related and unrelated)
sensitivity
ability of measures to DETECT effects