Lec2 - Ch5 Classical test theory models Flashcards
Classical Test Theory Models and Conceptual basis
1
Q
Reliability
- what is it, in regards to tests and scores?
A
- how much noise is there in a psychological test?
- it is a property of test scores, not of the test itself
> a test might have different psychometric properties for different kinds of respondents (i.e. it could be reliable for an age range but not the other)
> therefore, each set score has some level of reliability
2
Q
what is the COTAN
A
- committee that evaluates psychological tests in the Netherlands
- it is part of the NIP (Netherlands Institute of Psychology)
3
Q
how does the COTAN differentiate tests?
A
- test used for high-impact inferences at individual level
> very important; big consequences if mistake
> e.g. personnel selection, diagnosis of learning disabilities… - test used for less impact inference at individual level
> descriptive use, less consequences
> e.g. study/therapy progress, career choice test, … - test used at group level
> e.g. costumer team satisfaction, student evaluation, comparing groups
4
Q
high-impact inferences tests
- reliability rules
A
- good: 0.9 or larger
- sufficient: between 0.8 and 0.9
- insufficient: smaller than 0.8
5
Q
less impact inferences tests
- reliability rules
A
- good: 0.8 or larger
- sufficient: between 0.7 and 0.8
- insufficient: smaller than 0.7
6
Q
group level tests
- reliability rules
A
- good: 0.7 or larger
- sufficient: between 0.6 and 0.7
- insufficient: smaller than 0.6
7
Q
what is the aim of behavioural science?
A
- it strives to quantify the degree to which differences in one variable are associated with differences in other variables
- these differences have to be measured accurately, hence reliability
8
Q
What are the assumptions that testing is based on?
A
- behavioural differences among people exist
- differences have important implications
- they can be measured with precision
9
Q
what is Classical test theory?
A
- it is a measurement theory
- it explains reliability and it shows how to measure it
10
Q
What is the central idea of classical test theory?
A
Every test taker has a true score on a test
11
Q
what is reliability according to the classical test theory?
A
- Extent to which differences in respondent’s observed scores are consistent with differences in true scores
- it derives from observed scores, true scores and measurement error
12
Q
What are the two main assumptions of Classical Test Theory?
A
- observed scores are true scores plus measurement error
- measurement error is random (affects everybody but it is not systematic)
> likely to increase or decrease any particular score at random - Xo = Xt + Xe
13
Q
What are the implications of the assumptions of classical test theory? (consequences)
A
- mean of the measurement error is equal to zero
> because a non-zero mean would make the error systematic (the error cancels itself out) - the correlation between true score and error is equal to zero
> because the mean is zero for all error values - observed score variance = true score variance + error variance
14
Q
Observed scores
A
- value obtained from measuring a characteristic in a sample of individuals
- true score + measurement error
> (it can be seen as a composite score)
15
Q
True score
A
- Score that you would get using a perfect measurement instrument
- “real amount” of the characteristic you are measuring
- average score that a participant would obtain if they completed the test an infinite number of times
16
Q
Measurement Error
A
- Influences that create random noise in the observed score
- it creates inconsistencies between true and observed scores
> e.g. distraction, not precise meter, .. - it is impossible to know all the sources of measurement error and noise
- we must differenciate to which extent differences in scores are attributable to real differences in the trait or to random external influences
17
Q
what is the mean measurement error in a test?
A
- always 0
- it is independent of the individual’s true scores
- inflates or deflates respondents’ scores randomly, therefore it cancels itself out
- error scores are uncorrelated with true scores (r=0)
- see picture 1 for effects of measurement error
18
Q
Variance of error scores
- how to calculate it
- what it represents
A
- see picture 2
- it represents the degree to which error affected different people in different ways
> high degree of error variance indicates the potential for poor measurement
19
Q
how do you calculate the variance of observed scores?
A
- see picture 3
- variance of observed scores = variance of true scores + variance of error scores
> variability in observed scores will be larger than variability in true scores
