Psychological assessment Flashcards
A test is VALID if…
it accurately measures what it says it measures
A test is RELIABLE if…
it has the property of consistency in measurement
A test must be VALID for it to be RELIABLE T/F
FALSE
It is is the other way around.
Test must be RELIABLE in order to be VALID (necessary but not sufficient)
But not all reliable tests are VALID
Who is associated with classic test theory on reliability
Spearman (early 20th C)
First fundamental equation of Classical Test Theory?
observed score = true score + error
An example of an error that is endogenous to the test taker (to the test taker)
The person taking the test had a bad dream the night before
An example of an error that is exogenous to the test taker (to the test taker)
The psych recorded the wrong result
Assumptions of Classical Test Theory (4)
- Expected value of error is 0
- Errors do not correlate with each other
- Errors do not correlate with true scores (true score is unrelated to the direction of the error)
- Expected value of test is qual to the true score
SECOND fundamental equation of classical test theory
variance of the observed test results = variance of true scores + variance of errors + 2(covariance of true scores and variance)
The latter score is always ZERO
THIRD fundamental equation of classic test theory
Reliability = theoretical reliability coefficient
OR
Reliability = variance of true scores/variance of observed scores
OR
Reliability = Signal/signal + noise
What are the four ways of thinking of reliability?
- ratio of true score variance to observed score variance (MOST COMMON)
- Lack of error variance
- Squared correlation between observed scores and ten scores
- lack of correction between observed scores and error scores
What were the three forms of reliability he talked about
T P S
T P S
- Test-retest reliability
- Parallel forms reliability
- Split-half reliability
T P S
What are the problems for measuring test-retest reliability?
- Carryover affects
- True score may change between test and retest
- Participants might fail to return for the retest
What are the problems for measuring test-retest reliability?
- Hard to know whether the two forms are really parallel
2. Might not really ix the problem of carryover effect
What was Cronbach famous for?
The 1951 paper outlining a famous approach to reliability
What does Cronbach’s method provide?
A (conservative) lower bound estimate of reliability
What is Cronbach’s method, loosely?
The mean of all possible split-half reliabilities, scaled to the full test instead of the half test.
What is the standard error of measurement?
The standard deviation of the error
What is the equation for the standard error of measurement?
SEm = Sx * the square root pf 1-Rxx
Sx = standard deviation of observed scores
What did Kelley give us?
A way of predicting a true score
How do we predict someone’s true score, using the Kelley regression formula?
predicted true score = reliability coefficient x observed score + (1-reliability coefficients) x population mean
What do you use to calculate the standard error of estimation?
For determining CIs for predicted true scores
What is the Spearman-Brown prophecy?
A formula that you can use to estimate the increased reliability of a test resulting from an expanded number of questions
How do we improve reliability of tests
Increase
What is the tripartite division of validity?
- Criterion
- Content
- Construct
What is Criterion validity?
Compared favourably against a criterion
parallel or predictive
What is Content validity ?
extent to which test contents reflect the full domain of the construct it is supposedly assessing.
What is Lawshe about?
Approach for thinking about content validity
Content Validity Ratio
What is Construct validity?
Reflects the construct it is meant to be reflecting
What is sensitivity?
A test’s ability to correctly detect POSITIVE cases
Se = a/a+c
Where a is a test + criterion positive result and c is test negative and criterion positive
Correctly identifies sick people
What is Specificity?
A test’s ability to correctly detect NEGATIVE cases
What is positive predictive power? (PPP)
Probability a positive test result indicates a positive case
How worried you should be
What is negative predictive power? (NPP)
Probability a negative test result indicates a negative case
How happy should you be (Nobel peace prise)
Prevalence
What’s the probability a random person has the disease
Who came up with Multitrait-multimethod matrices?
Campbell and Fiske (1955)
