week 5 Flashcards
What are psychometrics
A collection of techniques for evaluating the development and use of psychological measures
What is reliability
free as possible from random error
What is validity
Does what it says on the tin
What is unidimensionality
measuring just the one thing we want to measure, or have we ended up measuring other things too
What is discrimination
how well do our items distinguish between levels of the thing we’re measuring
What is equivalence
Does the measure perform the same way for different groups of people
what is norm-referencing
How are scores distributed in the population
How are they standardised
rigorously tested for validity and reliability
Norm-referenced: compare scores against population norms
What are errors
It is unlikely any psychological measure will be 100% accurate
Observed score= true score plus error
Errors can be random in nature
Errors can be systematic in nature
How do we reduce random error
repeat measurement and average them
although this isn’t simple for psychological variables
How do we reduce systematic errors
Use multiple measurements each with different downsides
That way the variable of interest is measured consistently, but the noise is not
This is why psychological instruments tend to take he form of questionnaires rather than single questions
What is reliability
a measure is reliable if it is accurate and consistent
What is validity
A measure is valid if it measures what you think it is measuring
What are the types of reliability
Test-retest
parallel form reliability
Internal consistency
What is test-retest and strengths and weakness
Re doing the test to see if results were the same
Strengths: demonstrates the measure is temporally stable
weakness: based on a total score, what about emotion or motivation, How long between testing sessions
What is parallel forms of reliability
Two alternative forms of the same standardised measure
reduces risk of learning effects when evaluating reliability over time
What is internal consistency
a subscale should be unidimensional
So we look at whether every item on the subscale measures this dimension
If first 5 items in a scale are measuring the same thing, responses to these items will be highly correlated
Internal consistency is a way to look at this across items
Cronbach’s alpha
A number that reflects two things: mean correlation between items in a subscale and number of items in a sub scale
Half split technique
another way of quantifying internal consistency. Compare scores across two halves of a measure
Internal consistency: strength and weakness
Strengths: its essential poor internal consistency can only be due to items measuring different things
Weakness: if you increase the number of items, you increase Cronbach’s alpha- so exercise with very brief or very long measures
Extremely high Cronbach’s alpha values might be bloated too narrow a range of questions was asked
Inter-rater reliability
Largely used on coding of observational data: could be somewhat subjective, or somewhat objective
Internal validity
can the causal relationship that we established be explained by others?
External validity
can we generalise to other situations, other populations
Construct validity
How well we are measuring what we want to measure
Translation validity
is the operationalisation a good reflection of the construct
Criterion validity
How well does the measure agree with some external standard
On the face of things
does the instrument appear to measure the construct
Content
to what extent do the items actually represent the whole of the construct dimension that we are trying to measure
What do standardised measures do
give clear instructions
give instructions on how the measure give instructions on how the measure is administered
Predictive validity
does a score on the measure predict the value of another variable in the future
Concurrent validity
does the measure now correlate with information from a related measure
Convergent validity
does the measure correlate with another variable that it should theoretically be related to
Discriminant validity
does the measure correlate with a conceptually unrelated construct
Advantages of using standardised measures
Rigorous design process
Validity and reliability repeated tested
test have descriptive statistics for population norms which you can use to compare with your own
