Psychometrics and Test Construction

Question 1

Classical Test theory

Answer 1

Foundational theory for understanding reliability and validity of test scores

Question 2

Item Response Theory

Answer 2

Framework f or developing, evaluating, and scoring assessments. Addresses some limitations of classical test theory. More accurate and precise item characteristics

Question 3

Item characteristic curve

Answer 3

Concept from Item response theory. Describes the r/s between an individuals level on the latent trait being measured and their probability of providing a correct response to a specific test item

Question 4

Item discrimination

Answer 4

Refers to the extent to which a test item can differentiate between individuals with different levels of the latent trait being measured.

Question 5

Construct validity

Answer 5

The extent to which the measure assesses the domain, trait, or characteristic of interest (study habits, honesty, sympathy)

Question 6

Content Validity

Answer 6

Extent to which a test or assessment instrument adequately measures the intended construct or trait of interest

Question 7

Criterion-related validity

Answer 7

Demonstrates its effectiveness in predicting criterion or indicators of a construct, such as when an employer hires new employees based on normal procedure like interview, education, or experience

Question 8

Incremental Validity

Answer 8

Whether a new measure or measure of a new construct adds to an existing measure or set of measures with regard to some outcome present or future. Incremental validity is evident if the new measure adds statistical significance and can be evaluated in multiple regression and discriminant analyses.

Question 9

Ecological validity

Answer 9

Assesses the extent to which the findings from research studies accurately reflect real-world scenarios

Question 10

Inter-rater reliability

Answer 10

Consensus of scores given by various measures

Question 11

Internal consistency reliability

Answer 11

Assesses the extent to which items within a test or scale consistently measure the same underlying construct or dimension.

Question 12

split half

Answer 12

Form of internal consistency reliability
method used to assess the reliability of a test or scale. Divides the test into two halves and compares the scores on each half versus analyzing individual items

Question 13

Test-retest reliability

Answer 13

Test administered twice, to gauge the stability of the test over time to the same group

Question 14

Alternate forms reliability

Answer 14

The correlation between different forms of the same measure when the items of the two forms are considered to represent the same population of items

Question 15

Factor analysis

Answer 15

identifies patterns in the r/s among variables, identifying underlying dimensions or factors that explain the patterns of correlations

Question 16

Standard error of measurement

Answer 16

A measure of how much measured test scores are spread around a “true” score.

Question 17

Standard error of the difference

Answer 17

estimate the variability or uncertainty associated with the difference between two scores or measurements (comparing two groups or two sets of scores)

Question 18

Standard error of the estimate

Answer 18

Measure of the accuracy of predictions. Used when trying to predict what score a person will obtain on a test, depends on the criterion variable, can check accuracy of predictions

Question 19

WAIS

Answer 19

Wechsler Adult Intelligence Scale - widely used intelligence scale designed to measure cognitive abilities in adults and older adolescents

Question 20

WMS

Answer 20

Wechsler Memory Scale - neuropsychological assessment designed to assess various aspects of memory functioning in individuals ages 16-90

Question 21

PHQ

Answer 21

Patient health questionnaire - diagnostic tool used to screen for and assess the severity of depressive disorders and other mental health conditions

Question 22

IQ Scores (mean, SD, % of cases within 1/2 SD’s)

Answer 22

mean - 100, SD - 15, approx 68% of the population falls within one SD of the mean

Question 23

Risk factor correlations

Answer 23

R/s between different factors that contribute to the likelihood or probability of a negative outcome or event occurring (health, mortality)

Question 24

Discriminant Validity

Answer 24

Tests whether concepts or measurements that are not supposed to be related are actually unrelated (construct)

Question 25

Convergent Validity

Answer 25

The extent to which two measures that assess similar or related constructs correlate with each other (construct)

Question 26

Concurrent validity

Answer 26

a high correlation of the measure with other indices of the same construct (criterion-related)

Question 27

Predictive Validity

Answer 27

The correlation of a measure at one point in time with performance on another measure or criterion at some point in the future (criterion-related)

Question 28

Parallel-forms reliability

Answer 28

Assesses the degree to which test scores are consistent when there is a variation in the methods or instruments used. This allows inter-rater reliability to be ruled out.

Question 29

Chronbachs Alpha

Answer 29

Form of internal consistency reliability
ranges from 0-1 with higher values indicating greater internal consistency reliability based on the average correlation within the test