Psychometric Models: CTT vs IRT

Question 1

Latent variables

Answer 1

Latent variables= constructs are unobserved, hidden or latent variables inferred from the data collected on related observable variables

SEM= multivariate stats analysis technique used to analyse structural relationships among observable & unobserved (latent) variables
Implies a structure for the covariences between the observed variables

Question 2

Latent variable models

Answer 2

The relationship between the observable & the unobservable quantities is described by a mathematical function

Question 3

Classical test theory

Answer 3

Behavioural perspective

Measures the overall score on a test
Manifest behaviour is the unique reason representation of a construct, with no consideration to latent traits

Assumes the existence of the measurement error
Therefore aims to elaborate strategies (statistics) to control or evaluate the magnitude of error
Unit of analysis is the whole test (item sum or mean)

Question 4

Item response theory

Answer 4

Cognitive perspective

The answer a subject gives to an item depends on his or her level on the latent trait, the magnitude of his or her theta

Proposes the validation of items & not of tests
This favours the composition of large groups of independent items that can be used to create or customise different tests for different purposes

Question 5

Unidimensional IRT

Answer 5

Premise that the interactions of a person with test items can be adequately represented by a mathematical expression containing a single parameter describing the characteristics of the person

Question 6

Assumptions of unidimensional IRT

Answer 6

1) unidimensionality= a single latent trait variable is sufficient to explain the common variance among item responses
2) local independence= the response of any person to any test item is assumed to depend solely on the persons single parameter & the items vector of parameters, LI is evidence for unidimensionality if the IRT model contains person parameters on only 1 dimension

Question 7

Implications of local independence

Answer 7

Probability of a collection of responses can be determined by multiplying the probabilities of each of the individual responses

Question 8

Assumptions (3+4) of unidimensional IRT

Answer 8

3) the characteristics of a test item remain constant over all of the situations where it is used
4) monotonocity= probability of correct responses to the test item increases of does not decrease as the locations of examinees increase on the coordinate dimension

Question 9

Unidimensional IRT; understanding logistic regression

Answer 9

Linear regression= predicted Y can exceed y=0–1 range

Logistic regression= predicted Y lies within y=0-1 range

Question 10

Rasch model

Answer 10

Different approach to conceptualising the relationship between data & theory

Rasch= data fits the model 
IRT= best model that fits the data 

Rasch model not altered to suit data, method of assessment changed
Best estimate of the ability parameter for a person can be derived from his raw score only (sufficient statistic)
Discrimination parameter is set to 1.0

Question 11

Item information function

Answer 11

Can be used to predict the scores of examinees at given ability levels

If the amount of info is small, means that ability can not be estimated with precision about the estimates will be widely scattered about the true ability

Item info function depends on the item parameters a,b & c

Question 12

Polytomous models

Answer 12

Category response curves

e.g. likert scales

Question 13

Multidimensional IRT models

Answer 13

Item response surface=
Construct 1= intrinsic
Construct 2= extrinsic

Question 14

Estimation models

Answer 14

Likelihood function

Question 15

Model fit

Answer 15

Goodness of fit criteria to detect items that do not fit the specified response model

All models are incorrect in the sense that they provide incomplete representations of the data to which they are applied

A model doesn’t need to be perfect but fit the data well enough to be useful in guiding the measurement process

item response models referred to as strong models as underlying assumptions are stringent thus less likely to be met with tear data

IRT requires heterogenous & large examinee sample to insure proper item parameter estimation

Question 16

Parameter invariance

Answer 16

Reference & focal line on graph vary a lot

Question 17

CTT evaluation

Answer 17

1) standard error of measurement applies to all scores in particular pop
2) longer tests more reliable than shorter
3) test scores obtain meaning by comparing their position in a norm group
4) unbiased assessment of item properties depends on having representative samples

Question 18

IRT evaluation

Answer 18

1) standard error of measurement differs across scores but generalised across pops
2) shorter tests can be more reliable than longer ones
3) test score obtain meaning by comparing their distance from items
4) unbiased estimates of item properties may be obtained from unrepresentative samples