Choosing the best selection tool Flashcards
What is reliability?
How precise your measurement is in regard to consistency and repeatability
Which 3 reliability errors are there?
- Test development: if you develop a personality test and one item could be interpreted in multiple ways
- Test administration: keep environment and examinator constant, candidate (mood)
- Test scoring: if the person who evaluates the test makes a mistake
What is the test-retest reliability coefficient?
It is a measure of stability
→ The same test is administered at two different times to the same group of people
→ Use for stable constructs
What is the parallel forms reliability coefficient?
A measure of equivalence
→ Two different forms of the same test are administered to the same group of people
→ Have two different tests: one already established and one newly developed
What is the internal consistency reliability coefficient?
A measure of how consistently each item measures the same underlying construct
→ Use split-half, KR-20 or coefficient alpha
→ Single test administration
What is the inter-rater reliability coefficient?
A measure of agreement
→ Have two or more raters rate the same behaviour and then determine the agreement
What is the coefficient alpha / Cronbach alpha?
Every item as a test half and look at the correlation of each item. A high alpha indicates that a large portion of the variance in the test is attributable to the same underlying concept
→ ≥ .70 for low-stakes
& ≥ .80 for high-stakes decisions
→ Similar to using a weighing scale at home, the more often you measure it, the closer it is to the true value
What is validity?
Validity looks at how accurate your measurement is and is valid to the degree that it represents what you are trying to measure
→ e.g. if interested in job performance, does the test predict job performance?
What is construct validity?
Comprises the evidence supporting the trustworthiness of score interpretation in terms of underlying constructs.
What are the two threats of construct validity?
- Construct underrepresentation: measurement is too narrow and fails to include important parts of the construct
→ e.g. measuring intelligence: using only mathematics but this undermines e.g. verbal intelligence - Construct irrelevant variance: measurement is too broad, measuring variance associated with other constructs
→ e.g. measuring mathematical abilities: math questions that include text-based questions also measure verbal intelligence
What is internal structure validity?
The appraisal of theoretically expected patterns of relationships among items
→ Number of dimensions: e.g. LAQ are all five constructs actually explained by the test?
→ Group differences: are they all the same for subgroups?
What is external structure validity?
The appraisal of theoretically expected patterns of relationships between test scores and other measures.
→ Convergent validity: correspondence between measures of the same construct. Measures that should correlate with each other
→ Discriminant validity: distinctness from measures of other constructs. Measures that should not correlate with each other
What is criterion (predictive) validity?
The extent to which a test can predict scores on some criterion measures
What is incremental validity?
Whether a test adds something to the prediction of a criterion measure when you are already using a different test
→ e.g. the LAQ did not add anything extra when already using the HEXACO personality test
