Chapter 5 - Test 3 Flashcards
Validity
Truth (Accuracy of measurements, assumptions, …)
Reliability
Consistency (Accuracy of measurements, assumptions, …)
You can have with no, but you cannot have without .
reliability, validity, validity, reliability
Reliability Practices
- Psychological Tests (may already be available)
- Test-retest
- Alternate form/Parallel Form
- Internal Consistency (Inter-item)
a. split half (odd/even)
b. item/total - Inter-rater/Inter-judge/Inter-observer
Test-Retest
- static constructs (not changing)
- possible history effects (practice effects)
may improve
Alternate Form/Parallel Form
At the same time:
* make two tests (with the same mean and standard deviation)
* Hard to make “Equivalent” tests
Internal Consistency
each item can be considered a mini test, so…(use correlation matrix for measuring each item = average inter-item correlation
Split-half
- tests you compare are shorter the test you will use
- odd/even split (or any way to split them in half.)
- Cronbach’s alpha - all possible split test reliabilities.
nothing less than .70 RULE OF THUMB
Cronbach’s alpha
All possible split test reliabilities
Inter-judge/Observer/rater
*Controls consistency
provides a measure of the degree to which two judges concur in their respective sorting of N items into K mutually exclusive categories.
Cohen’s Kappa
External Validity
ability to generalize
Internal Validity
ability to determine cause/effect
Construct Validity
How well can you measure the construct you are interested in
* true score vs. measurement error (real value and fuzziness - errors)
* Observed score
Types of Constructs
- Face validity
- Divergent Validity (Discriminant)
- Convergent Validity
- Content Validity
- Predictive Validity (criterion validity)
- Concurrent Validity (criterion validity)
- Logical Argument
Face Validity
Does it seem to measure the construct?
Ex: Smith & Sons Welding Job Application
- Do you have any pets? (wrong)
- How many years have you been welding? (correct)
Divergent (Discriminant) Validity
Does it not correlate highly with a similar but different construct?
(tell apart one construct from another)
Ex: IQ Test ——–>Level of education
Convergent Validity
Does it correlate with another measure of the same construct?
Ex: Stanford-Binet IQ Test—> WAIS IQ Test (comparison to another test)
Content Validity
Do we sample from the universe of elements that compose the construct?
Ex: American History Test
(Not asking all the questions on only one subject)
Predictive Validity
Does it predict a future construct?
Ex: SAT Scores ——–> College Success
Concurrent Validity
Does it measure a current construct?
Ex: SAT Scores ——–> High School GPA
Logical Argument
Not empirical based
Ex: Does purchasing behavior translate into customer preference?
Coke sales are People like
higher than Pepsi —–> Coke better
External Validity
ability to generalize
Internal Validity
ability to determine cause and effect
construct validity
how well you can measure the construct you are interested in
construct
true score vs measurement error
observed score
Types of constructs
- Face validity
- Discriminant validity (divergent)
- convergent validity
- content validity
- predictive validity > criterion validity
- concurrent validity > criterion validity
- logical argument
Face Validity
Does it seem to measure our construct?
Example: Smith & Sons Welding Job Application
1. Do you have any pets?
2. How many years have you been welding? > correct
DEPENDS WHEN NEEDED*
Divergent Validity (DIscriminant)
Does it correlate with another measure of the same construct?
(comparison to another test)
Example: Stanford Binet IQ test»_space;»>WAIS IQ Test
Content Validity
Do we sample from the universe of elements that compose the construct?
Example: School Exams (varying questions not focusing on one subject only.
Predictive Validity
Does it measure a future construct?
Example: SAT Scores»_space;»»College Success
Concurrent Validity
Does it measure a current construct?
Example: SAT Scores»_space;»»High School GPA
Logical Argument
Example: Does purchasing behavior translate into customer preference?
Coke sales are People like
higher than Pepsi Coke better
NOT EMPIRICAL BASED*
