Problems of Measurement Flashcards
How can indirect experience contribute to empirical statments?
Indirect experiences can be based on other experiences through inferences we make
Know the definitions of the five types of statements (empirical, analytic, value, attitude, metaphysical) and be able to identify them
Empirical: information is obtained through the experience of our senses, through observations
Analytical: assert something about the meaning of words, not the observable world. They are true or false.
Value: express some positive or negative evaluation of something or someone
Attitude: express feeling or what they are thinking about something, but little to no observation about that something
Metaphysical: asserting that something cannot be observed with out senses, lack empirical meaning.
What is the verifiability principle and how does it relate to operational definitions in research?
Verifiability principle: empirical statement tells us what sense experiences people would have if the statements were TRUE
Operational definitions: defining your measures and what they mean, so people can understand what you are doing
Words used in the statement have to have the same meaning about experience for everyone who wants to verify
What is the role of falsifiability in empirical statements? How can a belief system become unfalsifiable?
Falsifiability principle: an empirical statement should also tell us what sensory experience we should have if the statement were FALSE
Belief systems can undermine falsifiability by relying on excuses so they cannot be falsified
Understand how empirical statements are not necessarily true or immediately verifiable
Truth or falseness depends on observations made and the observed reality may or may not confirm these
immediately verifiable: may not be possible to make the necessary observations, but we should still know what observations would be necessary if it were possible. We do not need to prove statements for them to be empirical.
Be aware how slight changes in wording can re-classify one kind of statement to another
Can change non empirical, measurable, statements into measurable statements. e.g. going from “I love you” to “I spend more time with you than anybody else”.
Be able to define, give examples of, and identify the follow properties of measurement, random error/noise
Error values fluctuate randomly around the underlying true value of the variable. Repeated measure can cancel out the mean random error
Be able to define, give examples of, and identify the follow properties of measurement, systematic error/bias
An error that distorts measurements consistency by a fixed amount from the underlying true value/ If experimenter bias impacts one group more than this can be critiqued
Be able to define, give examples of, and identify the follow properties of measurement, reliability/precision
An index which measures how well random noise has been controlled
Be able to define, give examples of, and identify the follow properties of measurement, internal consistency
The more reliable the measure the better the statistical analysis
Be able to define, give examples of, and identify the follow properties of measurement, test-retest reliability
Test at one timepoint then again at another and compare
Be able to define, give examples of, and identify the follow properties of measurement, expectancy effects
Research process by the experimenter unconsciously influencing responses
Be able to define, give examples of, and identify the follow properties of measurement, selection bias
The kind of participants selected may exaggerate or diminish results
Be able to define, give examples of, and identify the follow properties of measurement, testing effects
The process of testing may change people and give artificial results
Be able to define, give examples of, and identify the follow properties of measurement, demand characteristics
Participants figure out what the study is testing and play along
Be able to define, give examples of, and identify the follow properties of measurement, response bias
Participants may give biased responses in the yes direction, avoid extremes, or try to make themselves look good
Be able to define, give examples of, and identify the follow properties of measurement, internal validity/accuracy
How well you measure what you want to measure and not something else, depends on how well systematic error and confounds have been controlled for
Be able to define, give examples of, and identify the follow properties of measurement, convergent validity
The used scale should correlate better with things related to it. e.g. depression should correlate with mood swings
Be able to define, give examples of, and identify the follow properties of measurement, discriminate validity
your scale should correlate better with what you are actually measuring than something else. e.g. depression and not anxiety
Be able to define, give examples of, and identify the follow properties of measurement, external validity
How well the measure generalises to a variety if real world situations
Be able to define, give examples of, and identify the follow properties of measurement, measurement invariances
Same relationship among items in different populations
Know how measurement and manipulations in psychology can be improved beyond the current standards and the roles of manipulation checks and testing
Measurement in psychology: done on the spot, making up items, with no attempt to test their reliability or internal validity beforehand
Manipulations in psychology: are often not tested, either with a manipulation check in the study or with a separate pre-test
To improve these we should do the opposite
Be familiar with the evidence and argument for widespread poor measurement practices in psychology
Evidence: most measurements used in experiments are lacking evidence, according to Barry et al., (2014), they reported 40% to 93% or measures lacked validity evidence
Weidman et al., (2017) reported that among 356 measurement instances coded in the reviews of emotions research, 69% included no reference to prior research.
Argument: this occurs because of a lack of transparency
What is the argument as to why there is widespread poor measurement practices in psychology?
Due to a lack of transparency