Chapter 1: Classical Test Theory Flashcards
What does CTT represent?
Classical test theory (CTT) is the foundational theory of measurement of mental abilities. At its core, CTT describes the relationship between observed composite scores on a test and a presumed but unobserved “true” score for an examinee.
How does CTT hold up against other modern theories of measurement?
Modern theories of measurement, such as IRT (item response theory), do not obviate CTT or even contradict it; rather, they extend it although there are important distinctions in both the underlying philosophies and in the statistics employed for implementation.
Name a primary feature of CTT and why it is necessary
A primary feature of CTT is its adherence to learning theories that follow notions of classical and operant conditioning (e.g., behaviorism, social learning theory, motivation). CTT presumes extant a domain of content apart from any particular examinee, although – significantly – the domain is not reified; it remains an abstraction. This perspective places CTT outside cognitivist theories of learning (e.g., information processing, constructivism). Thus, for application of the theory, the domain is defined anew in each appraisal. For example, if “reading” is the domain for an appraisal, “reading” must be defined for that specific assessment. In another assessment “reading” will have a slightly different meaning. Hence, in CTT, no two independent tests are identical, although strictly parallel forms for a given assessment may be developed.
How is the observed score cosidered in the CTT framework?
In the CTT framework, an individual’s observed score on a test is considered to be a random variable with some unknown distribution. The individual’s true score is the expected value of this distribution.
What name is given to the discrepancy between the individual’s observed score and true score?
Measurement error
How does CTT build two central definitions based on the elements?
(1) the true score tgp of a person p on measurement g is the expected value of the observed score Xgp;
(2) the error score Egp which is the difference between the two elements (i.e., observed score and the true score, Xgp−tgp)
What does Fgp denote?
Under CTT, tgp is a constant yet unobserved value, and Xgp is a random variable that fluctuates over repeated sampling of measuring g. This fluctuation is reflected by a propensity distribution Fgp for that person p and measurement g.
From this stand point the mathematical model for CTT can be deduced, and consists of two equations.
Name these
Tgp = E(Xgp)
Egp = Xgp - Tgp
However, in most cases, researchers are interested in the traits of a population of people rather than in the trait of a fixed person p.
What implications does this have?
Therefore, any person p from that population can be considered a random sample. The notation Xg presents a random variable defined over repeated sampling of persons in a population, which takes a specific value xg when a particular person is sampled. Similarly, Γg is a random variable over repeated sampling of persons in a population, which takes a specific value tg when a particular person is selected. Finally, Egis random variable representing the error score. Under this construction, Lord and Novick (1968) had the theorem that Xg = Γg + Eg. Without loss of generality, the subscript g is omitted when only one measurement is considered:
XE=Γ+E
How are the variables classified in the equation X=Γ+E?
all the three elements are random variables. In CTT they are called “random variables,” although in the more general probability theory they are classified as stochastic processes.
Name the three assumptions of CTT
CTT as a theory requires very weak assumptions. These assumptions include:
(a) the measurement is an interval scale
(b) the variance of observed σ^2x is finite
(c) the repeated sampling of measurements is linearly, experimentally independent.
Under these assumptions of CTT, what 8 properties have been derived?
- The expected error score is zero;
- The correlation between true and error scores is zero;
- The correlation between the error score on one measurement and the true score on another measurement is zero;
- The correlation between errors on linearly experimentally independent measurements is zero;
- The expected value of the observed score random variable over persons is equal to the expected value of the true score random variable over persons;
- The variance of the error score random variable over persons is equal to the expected value, over persons, of the error variance within person (i.e., σ^2(Xgp ));
gp - Sampling over persons in the subpopulation of people with any fixed true score, the expected value of the error score random variable is zero;
- The variance of observed scores is the sum of the variance of true scores and the variance of error scores; that is:
σ^2x = σ^2r + σ^2E
