Psychometrics and Test Construction Flashcards
Classical Test theory
Foundational theory for understanding reliability and validity of test scores
Item Response Theory
Framework f or developing, evaluating, and scoring assessments. Addresses some limitations of classical test theory. More accurate and precise item characteristics
Item characteristic curve
Concept from Item response theory. Describes the r/s between an individuals level on the latent trait being measured and their probability of providing a correct response to a specific test item
Item discrimination
Refers to the extent to which a test item can differentiate between individuals with different levels of the latent trait being measured.
Construct validity
The extent to which the measure assesses the domain, trait, or characteristic of interest (study habits, honesty, sympathy)
Content Validity
Extent to which a test or assessment instrument adequately measures the intended construct or trait of interest
Criterion-related validity
Demonstrates its effectiveness in predicting criterion or indicators of a construct, such as when an employer hires new employees based on normal procedure like interview, education, or experience
Incremental Validity
Whether a new measure or measure of a new construct adds to an existing measure or set of measures with regard to some outcome present or future. Incremental validity is evident if the new measure adds statistical significance and can be evaluated in multiple regression and discriminant analyses.
Ecological validity
Assesses the extent to which the findings from research studies accurately reflect real-world scenarios
Inter-rater reliability
Consensus of scores given by various measures
Internal consistency reliability
Assesses the extent to which items within a test or scale consistently measure the same underlying construct or dimension.
split half
Form of internal consistency reliability
method used to assess the reliability of a test or scale. Divides the test into two halves and compares the scores on each half versus analyzing individual items
Test-retest reliability
Test administered twice, to gauge the stability of the test over time to the same group
Alternate forms reliability
The correlation between different forms of the same measure when the items of the two forms are considered to represent the same population of items
Factor analysis
identifies patterns in the r/s among variables, identifying underlying dimensions or factors that explain the patterns of correlations
Standard error of measurement
A measure of how much measured test scores are spread around a “true” score.
Standard error of the difference
estimate the variability or uncertainty associated with the difference between two scores or measurements (comparing two groups or two sets of scores)
Standard error of the estimate
Measure of the accuracy of predictions. Used when trying to predict what score a person will obtain on a test, depends on the criterion variable, can check accuracy of predictions
WAIS
Wechsler Adult Intelligence Scale - widely used intelligence scale designed to measure cognitive abilities in adults and older adolescents
WMS
Wechsler Memory Scale - neuropsychological assessment designed to assess various aspects of memory functioning in individuals ages 16-90
PHQ
Patient health questionnaire - diagnostic tool used to screen for and assess the severity of depressive disorders and other mental health conditions
IQ Scores (mean, SD, % of cases within 1/2 SD’s)
mean - 100, SD - 15, approx 68% of the population falls within one SD of the mean
Risk factor correlations
R/s between different factors that contribute to the likelihood or probability of a negative outcome or event occurring (health, mortality)
Discriminant Validity
Tests whether concepts or measurements that are not supposed to be related are actually unrelated (construct)
Convergent Validity
The extent to which two measures that assess similar or related constructs correlate with each other (construct)
Concurrent validity
a high correlation of the measure with other indices of the same construct (criterion-related)
Predictive Validity
The correlation of a measure at one point in time with performance on another measure or criterion at some point in the future (criterion-related)
Parallel-forms reliability
Assesses the degree to which test scores are consistent when there is a variation in the methods or instruments used. This allows inter-rater reliability to be ruled out.
Chronbachs Alpha
Form of internal consistency reliability
ranges from 0-1 with higher values indicating greater internal consistency reliability based on the average correlation within the test
