Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Comps > Comps - Assessment (Reliability) > Flashcards

Comps - Assessment (Reliability) Flashcards

1
Q

Reliability

A

Reflects the degree to which differences in observed scores are consistent with differences in true scores

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Because true scores are __________, reliability is __________; it can only be __________, not actually calculated

A

theoretical, theoretical, estimated

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

In layman’s terms, what does reliability do?

A

It provides an index of score stability, giving you an idea of how consistently you are measuring something

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Reliability is what from validity?

A

Independent

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

If measurements are __________, then there is theoretically little __________

A

consistent, error

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What happens when observed and true score variance are equal?

A

The reliability coefficient is 0, meaning that there is no measurement error (“perfect” reliability)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Classical Test Theory (CTT)

A

The conceptual basis for the entire concept of reliability. It was pioneered by Spearman and focuses on overall test scores

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

CTT Equation

A

Observed Score (X) = True Score (T) + Error (E)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What is the goal of CTT in assessments?

A

To minimize the error component, ensuring maximum congruence between the observed score and the true score

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

CTT assessment examples

A

Woodcock-Johnson, standardized testing (GRE, SAT, ACT), Wechsler, Stanford-Binet, MMPI

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

CTT items must measure the following:

A

-Same latent variable
-Same scale
-Same degree of precision
-Same amount of error

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Tau-Equivalence

A

Adjacent model to CTT

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

How are CTT and Tau-Equivalence different?

A

Same assumption as CTT, with the exception that individual item variance can differ

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Tau-Equivalence items must measure the following:

A

-Same latent variable
-Same scale
-Same degree of precision
-Different amount of error

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Essential Tau-Equivalent Model

A

Same assumptions as tau-equivalent, but individual items can measure the same variable with different degrees of precision and different error

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What does the Essential Tau-Equivalent Model consider?

A

That the true score may be measured more accurately by one item than another

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Essential Tau-Equivalent Model must measure the following:

A

-Same latent variable
-Same scale
-Different degree of precision
-Different amount of error

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Which model of reliability is the most representative and practical?

A

Essential Tau-Equivalent Model

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Alternate Forms (method for estimating reliability)

A

Has 2 different forms that must measure the same thing, which is impractical in estimating reliability as it is hard to determine if the 2 forms are tau-equivalent. The correlation between Form 1 and Form 2 is an indicator of reliability

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

With Alternate Forms, how do we know that one true score is equal to the other?

A

If they really are equivalent, exposure to one form might impact your performance on the form. The person might be bored or remember the information from the first form. Very few tests make parallel alternate forms because it is difficult, expensive, and time-consuming

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Test-Retest Model (method for estimating reliability)

A

Take the test, then take the test again a second time

22
Q

What is a good timeframe to give a second test in the test-retest model?

A

2-8 weeks (want it to be long enough to minimize memory, practice, and learning effects while being short enough to prevent maturational developments or historical changes that would affect the true score)

23
Q

Two Assumptions (method for estimating reliability)

A

The true score remains stable across testing conditions and error variance is equal in both administrations

24
Q

Why might the true score not remain stable across testing conditions?

A

-State vs. trait (your trait should not change, but state while taking the test might)
-Carry-over effects, practice effects, maturation effects, interval between administrations
-Additional sources of error that might make performance better or worse
-Introduces sources of error that lower the reliability coefficient

25
Q

State

A

Mood, emotional state

26
Q

Trait

A

Personality, negative affectivity (predisposed to view the world negatively)

27
Q

Why might the error variance be equal in both administrations?

A

-If the testing conditions differ or the intervals are far apart
-Testing at the same time of day in the same testing conditions can reduce the likelihood of different error variance
-Increased likelihood of carry-over effects when you learn how to take a test as you take it

28
Q

Internal Consistency

A

Looks at inter-relatedness of test items and treats different parts of a test as different forms

29
Q

What does internal consistency assume?

A

Unidimensionality (you can only divide a test into different forms if all of the items are measuring the same thing)

30
Q

If items are strongly correlated with each other, what does that suggest?

A

That each item is measuring the same construct

31
Q

What are the two forms of internal consistency?

A

Split-Half Reliability and Coefficient Alpha (Cronbach’s Alpha)

32
Q

Split-Half Reliability

A

Items on a test are sorted into parallel subtests of equal length (evens vs. odds or down the middle). The correlation between the two scores is the reliability coefficient

33
Q

What does CTT say about shorter tests, regarding Split-Half Reliability?

A

They are less reliable, assumes that new items are equivalent to existing ones, and assumes that split halves are parallel with the same true score and same variance

34
Q

Coefficient Alpha (Chronbach’s Alpha)

A

Each item is considered its own subtest and it is grounded in the tau-equivalent model. It is the most widely used method for estimating reliability. The average of alpha is the reliability coefficient

35
Q

Diagnostically, what is a good way to think about Coefficient Alpha?

A

It is a quick way to know that the items are tapping into the construct being measured (e.g., the BDI has items measuring the construct of depression)

36
Q

Is Split-Half Reliability or Coefficient Alpha more accurate?

A

Coefficient Alpha

37
Q

Standard Error of Measurement (SEM)

A

An estimate of the amount of error inherent in an obtained score; directly reflects reliability

38
Q

What does higher SEM mean?

A

It is less reliable because there is a negative relationship

39
Q

Reliability and Clinical Diagnosis

A

Our diagnostic criteria are supposed to be an approximation of the natural phenomena that is a disorder. The quality of that relationship between what they have and what they are diagnosed with is based on sensitivity and specificity

40
Q

What is the problem with reliability of a clinical diagnosis?

A

The DSM diagnosis could be correlated with various other disorders because the DSM defines a diagnosis and not a disorder

41
Q

What are the three variances in diagnoses that are non-overlapping influences?

A

Disorder, contaminants, and random error

42
Q

Disorder

A

Variance among patient samples in characteristic related to the disorder that affects the diagnosis

43
Q

Contaminents

A

A portion is due to variance among the patients sampled in characteristics unrelated to the disorder but that affect the diagnosis

44
Q

Random Error

A

Factors not characteristic of the patient that impact the diagnosis (e.g., random fluctuations, errors by the diagnosticians, or lack of clarity in the diagnostic criteria)

45
Q

Reliability is measured and described via __________ and __________

A

sensitivity and specificity

46
Q

Sensitivity

A

The probability of correctly diagnosing true positives (i.e., am I accurately getting the diagnosis?)

47
Q

What does high sensitivity suggest?

A

Low type II error (miss)

48
Q

Specificity

A

The probability of correctly diagnosing true negatives (i.e., am I accurately ruling this out?)

49
Q

What does high specificity suggest?

A

Low type I error (false alarm(

50
Q

Type I Error

A

You think you found a significant effect when there really is not one

51
Q

Type II Error

A

You miss a significant effect that is really there

Comps (13 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions