Brainscape's Knowledge GenomeTM


Top Flashcards (Certified)
All Flashcards

PTPR Retake > L2: Classical Test Theory > Flashcards

L2: Classical Test Theory Flashcards

(107 cards)

Start Studying
1
Q

What is the central statistic in classical test theory?

A

The sum score

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

True Score - Definition

A

The score that would be obtained if a perfect measurement instrument was used

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Classical Test Theory - Core Assumptions

A

1) Observed score = true score + measurement error
2) Measurement error is random

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What are the implications of measurement error being random

A

1) mean m.err = 0
(it cancels out bc it is random)
2) there is no correlation between the true score & measurement error
3) all of the observed variance can be explained by the true score variance & measurement error (there are no other sources of noise)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Classical Test Theory - Definition

A

Measurement theory that defines the conceptual basis of reliability & outlines procedures from estimating the reliability of psychological test scores

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Measurement Error - Definition

A

Extent to which other characteristics contribute random noise to the differences in observed scores

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Reliability - Definition

A
  • Measure of whether something is consistent (stays the same)
  • Results are considered to be reliable if they are similar each time they are carried out using the same design, procedures, measurements
  • Extent to which differences in respondents observed scores are consistent with differences in their true scores
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What are two ways of defining reliability?

A

1) As a proportion of variance
2) As a proportion of shared variance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Reliability (as a proportion of variance) - Formula

A

True score variance/observed score variance

1 - (error score variance/observed score variance)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Reliability (as a proportion of variance) - Definition

A

The proportion of observed score variance that is attributable to true score variance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Reliability (as a proportion of shared variance) - Definition

A

The proportion of variance shared between the true scores & observed scores

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Reliability (as a proportion of shared variance) - Formula

A

(correlation observed+true)²

1 - (correlation observed+error)²

Note: squaring a correlation gives you the amount of variance shared by those variables

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What are the four test models of reliability?

A

1) Parallel Test Model
2) Tau Equivalent Test Model
3) Essential Tau Equivalent Test Model
4) Congeneric Test Model

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

List the test models of reliability from most to least restrictive

A

1) Parallel Test Model
2) Tau Equivalent Test Model
3) Essential Tau Equivalent Test Model
4) Congeneric Test Model

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Parallel Test Model - Assumptions

A

True Scores:
= mean
= variance

Observed Scores:
= mean
= variance

Error Scores:
= variance

Correlation:
= correlation true&observed

Reliability:
= R (test 1 & 2)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Parallel Test Model - Assumptions True Scores

A

= mean
= variance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Parallel Test Model - Assumptions Observed Scores

A

= mean
= variance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Parallel Test Model - Assumptions Error Scores

A

= variance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Parallel Test Model - Assumptions Reliability

A

= reliability

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

Parallel Test Model - Tests

A

1) Alternate Forms
2) Split-Halves
3) Test-Retest

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Tau Equivalent Test Model - Assumptions

A

True Scores:
= mean
= variance

Observed Scores:
= mean

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Tau Equivalent Test Model - Assumptions True Scores

A

= mean
= variance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

Tau Equivalent Test Model - Assumptions Observed Scores

A

= mean

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

Tau Equivalent Test Model - Assumptions Error Scores

A

none

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Tau Equivalent Test Model - Assumptions Reliability
none
26
Tau Equivalent Test Model - Tests
1) Alpha
27
Essential Tau Equivalent Test Model - Assumptions
True Scores: = variance
28
Essential Tau Equivalent Test Model - Assumptions True Scores
= variance
29
Essential Tau Equivalent Test Model - Assumptions Observed Scores
none
30
Essential Tau Equivalent Test Model - Assumptions Error Scores
none
31
Essential Tau Equivalent Test Model - Assumptions Reliability
none
32
Essential Tau Equivalent Test Model - Tests
Cronbach's Alpha
33
Congeneric Test Model - Assumptions
none
34
Congeneric Test Model - Assumptions True Scores
none
35
Congeneric Test Model - Assumptions Observed Scores
none
36
Congeneric Test Model - Assumptions Error Scores
none
37
Congeneric Test Model - Assumptions Reliability
none
38
Congeneric Test Model - Tests
Omega
39
Parallel Test Model - True Score Formula
Xt2 = Xt1
40
Tau Equivalent Test Model - True Score Formula
Xt2 = Xt1
41
Essential Tau Equivalent Test Model - True Score Formula
Xt2 = a + Xt1
42
Congeneric Test Model - True Score Formula
Xt2 = a +bXt1
43
Parallel Test Model - Observed Score Formula
Xo1 = Xt1 + Xe1 Xo2 = Xt1 + Xe2
44
Tau Equivalent Test Model - Observed Score Formula
Xo1 = Xt1 + Xe1 Xo2 = Xt1 + Xe2
45
Essential Tau Equivalent Test Model - Observed Score Formula
Xo1 = Xt1 + Xe1 Xo2 = a + Xt1 + Xe2
46
Congeneric Test Model - Observed Score Formula
Xo1 = Xt1 + Xe1 Xo2 = a + bXt1 + Xe2
47
What are the main methods of reliability estimation?
1) Alternate Forms 2) Test-Retest 3) Internal Consistency
48
What are the different tests in the alternate forms method?
2 alternate forms/versions of the same test
49
What test model does the alternate forms method follow?
parallel test model
50
What is the reliability in the alternate forms method?
reliability is the correlation (between the 2 test versions)
51
What are limitations of the alternate forms method?
- It is very difficult in practise to create two versions of a test that are unique yet still parallel - carryover effects
52
Carryover Effects - Definition
An effect of being tested in one condition on participants behaviour in later conditions
53
What are the different tests in the test-retest method?
the same person takes the same test on more than one occasion
54
What test model does the test-retest method follow?
Parallel test model
55
What is the reliability in the test-retest method?
Reliability is the correlation (of the two test taking occasions)
56
What are limitations of the test-retest method?
- Difficult to do for constructs that naturally fluctuate over time (change in true scores) - Carryover effects - People might not want to take the test a second time
57
What are the different tests that fall under the general internal consistency method?
1) Split-Half 2) Cronbach's Alpha 3) Omega
58
What are limitations of the general internal consistency method?
Carry over effects can cause there to be a correlation between the error scores of different items
59
What are the different tests in the internal consistency method?
(Blocks of) items are treated as separate tests
60
What test model does the internal consistency method follow?
Parallel test model OR essential tau equivalent model
61
What are the different reliability measures under Cronbach's Alpha?
- Raw Alpha - Standardized Alpha - KR20
62
What test model does Cronbach's Alpha follow?
Essential tau equivalent model
63
What are limitations of Cronbach's Alpha?
- assumptions are hardly ever met in reality - Cronbach's Alpha is lower bound to the reliability (it will underestimate the reliability)
64
What are the different tests in Cronbach's Alpha?
1) Raw Alpha 2) Standardised Alpha 3) KR20
65
When should you use Raw Alpha?
For tests with items that do not substantially differ in their variances
66
Raw Alpha - Consistency Index
Sum of all covariances amongst items (ΣC ᵢ ᵢ)
67
When should you use Standardised Alpha?
For tests with items that substantially differ in their variances, which causes the test scores to only reflect items with very high variances
68
Standardised Alpha - Consistency Index
Average of all correlations amongst items (r ᵢ ᵢ,)
69
When should you use KR20?
For binary items
70
KR20 - Consistency Index
Sum of item variances (Σpq)
71
What are the different tests in Omega?
Each item of the test is considered to be a separate test
72
What test model does Omega follow?
Congeneric test model
73
What is reliability in Omega?
Reliability is the: true score variance/observed score variance
74
What reliability estimation method is omega apart of?
Internal Consistency
75
What reliability estimation method is split-halves apart of?
Internal Consistency Method
76
What reliability estimation method is alpha apart of?
Internal Consistency Method
77
What are the different tests in split-halves?
Test is split into 2 parts
78
What test model does split-halves follow?
Parallel test model
79
What is the consistency index in split-halves?
Reliability is the correlation (between test halves)
80
What are the limitations of split-halves?
- Reliability is heavily influenced by the type of split done - It cannot be used for speeded tests, as you will almost always get a correlation close to 1.0. This is because response speeds are consistent throughout the entire test. - Other methods utilise more information about the test
81
What are factors that affect reliability?
1) Test Length 2) Sample Heterogeneity 3) Reliability of Difference Scores
82
How does test length influence the reliability of a test?
reliability will increase with more items added
83
How does sample heterogeneity influence the reliability of a test?
In homogenous samples, the reliability is lower because of lower true score variance. In heterogenous samples, the reliability is higher because of greater true score variance. Both of these are not desirable, as reliability should be a property of the test, and not a property of the sample being examined.
84
Reliability Generalisation Study - Definition
Study intended to reveal the degree to which a test produces differing reliability estimates across different kinds of research uses & populations (aka how sample characteristics affect the reliability of test scores)
85
Difference Score - Formula
posttest score - pretest score
86
What influences the reliability of difference scores?
The correlation between pretest & postest. If the correlation is very high, the reliability is low.
87
What is the difference score sensitive to?
The variance between pretest & post-test. However, this is not as relevant for pretest -posttest designs as the difference will never be that large
88
What are the approaches to true score estimation?
1) True score estimate is the summed item score 2) True score estimate is the summed item score, corrected for regression to the mean. The lower the reliability is, the more the true score estimate is corrected to be closer to the mean.
89
Regression to the mean - Definition
If one sample of a variable is extreme, the next sampling is likely to be closer to the mean.
90
What does standard error (sem) represent?
The average size of error scores
91
What is the relation between reliability & standard error?
The higher the reliability, the lower the standard error, and vice versa
92
Attenuation - Definition
lessening/weakening in the intensity, value, or quality of a stimulus. In terms of reliability, attenuation refers to the fact that the effect sizes/correlations of the observed scores will always be smaller than that of the true scores, due to including the measurement error.
93
What is the effect of reliability on statistical significance?
Reliability has a direct effect on statistical significance- With high reliability, larger observed effect sizes are possible, which increases the likelihood of a significant result.
94
Point estimate - Definition
Specific value that is interpreted to be the best estimate of an individuals standing on a particular psychological attribute
95
Internal Consistency - Definition
Degree to which differences amongst participants responses to one item are consistent with differences amongst their responses to other items on the test (how consistent the test items are with each other)
96
Item Discrimination - Definition
Degree to which an item differentiates people who score high on the total test from those who score low on the total test
97
Item-total Correlation - Definition
Degree to which differences amongst participants responses to the item are consistent with differences in their total test score. If the correlation is high, then the item is highly consistent with the total test scores.
98
Corrected Item-total Correlation - Definition
The consistency between an item and the other items on a test (correlation between responses to item one and the sum of all other items on the test)
99
How to interpret "Cronbach's Alpha if Item Deleted" on an Item-Total Statistics Table?
Items that increase alpha when dropped should be removed. However, only remove these items if the increase in reliability is deemed important enough.
100
How to interpret "Corrected Item-Total Correlation" on an Item-Total Statistics Table?
This table tells you the consistency between that item & other items on the test. Items with a high corrected item-total correlation also have high item discrimination.
101
When do you use the Discrimination Index?
When analysing the Internal Consistency of a test with binary items
102
What does the Discrimination Index show?
The proportion of high test scorers that answered the item corrected compared with the proportion of low test scorers that also answered the item correctly.
103
How do you interpret a Discrimination Index value?
Higher DI values are indicative of greater internal consistency, as high and low test scorers differ significantly in the likelihood of answering the item correctly
104
What is COTAN?
A Dutch committee involved in the evaluation of (new) psychological tests
105
What are the COTAN guidelines for high impact inferences at the individual level?
good = r ≥ .9 satisfactory = .8 ≤ r ≤ .9 insufficient = r ≤ .8 (between .8 & .9)
106
What are the COTAN guidelines for lower impact inferences at the individual level?
good = r ≥ .8 satisfactory = .7 ≤ r ≤ .8 insufficient = r ≤ .7 (between .7 & .8)
107
What are the COTAN guidelines for inferences at the group level?
good = r ≥ .7 satisfactory = .6 ≤ r ≤ .7 insufficient = r ≤ .6 (between .6 & .7)

PTPR Retake (6 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2025 Bold Learning Solutions. Terms and Conditions