Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

3006PSY > Module 3: Reliability and Validity > Flashcards

Module 3: Reliability and Validity Flashcards

1
Q

What is Reliability?

A

Consistency/stability of measurement.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What is Validity?

A

The extent to which a test measures what it is meant to measure.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What is the Classical Test Theory?

A

The assumption that the score of each test taker reflects their actual ability, plus a degree of measurement error.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What is the Equation for the Concept of Error?

A

Observed score = true score + sources of error.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What is Random Error?

A

Source of error caused by unpredictable fluctuations and inconsistencies of other variables in the measurement process.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is Systematic Error?

A

Source of error that is typically constant or proportionate to what is presumed to be the true value of the variable being measured.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What are the Four Sources of Error Variance?

A
  1. Test construction (sampling of content/poor item wording).
  2. Test administration (due to test-taker, test-environment, test-examiner).
  3. Test scoring and interpretation (objective vs subjective scoring).
  4. Other sources of error (sampling error, methodological error, social desirability bias/impression management).
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What is the Standard Error of Measurement (SEM)?

A

Estimates the degree of error associated with the observed score.
Equation: SEM = SD x square root of 1.0 - r.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What are the Confidential Intervals and SEM?

A

68% = 1 SEM.
95% = 2 SEM.
99% = 3 SEM.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What is Reliability Coefficient?

A

Index of reliability, a proportion that indicates the ratio between the true score variance on a test and the total variance.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

How is Variance Measured?

A

Standard deviation squared.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What are the Four Main Types of Reliability?

A
  1. Consistency across time (test-retest).
  2. Consistency across different versions (alternative/parallel forms).
  3. Consistency across individual items of a test (internal consistency).
  4. Consistency across scorers (inter-rater reliability).
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What is Test-Retest Reliability?

A

If a construct being measured remains unchanged, then similar scores will be observed at time 1 and time 2.
The longer time passes, this reliability coefficient can lower.
Subject to practice/familiarity effects.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What are Alternative/Parallel Forms?

A

Degree to which different versions of a test/measure correlative.
Expensive and time-consuming.
Avoids practice effects.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What is Internal Consistency?

A

Degree to which different items in a test correlate with each other.
Not content, just consistency (how similar they are?).

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What is Inter-Related/Inter-Scorer Reliability?

A

Degree to which two or more scorers correlate in measure.
Scoring criteria must be clear and judges should be well-trained.
Coefficient = cohen’s kappa (k).

17
Q

At a Minimum, What Should all Tests Have?

A

Content validity.
Internal consistency reliability.

18
Q

What is Ecological Validity?

A

Extent to which a study is realistic or representative of real life.

19
Q

What is Psychometric Soundness?

A

Evaluation of both reliability and validity.

20
Q

What is Face Validity?

A

Extent to which the scale looks like what it claims to measure.

21
Q

What is Content Validity?

A

Extend to which items represent the construct universe.

22
Q

What is Criterion-Related Validity?

A

How well the test correlated with external criteria or outcome (real world)?
Does it have predictive power?

23
Q

What are the Two Types of Criterion-Related Validity?

A

Concurrent: compare test score with criterion measured at the same time.
Predictive: compare score with a criterion ar some point in the future.

24
Q

What is Construct Validity?

A

Testing the correlation between the construct and other theoretically related constructs (nomological network).

25
Q

What are the Two Ways of Demonstrating Construct Validity?

A

Convergent: tests should correlate strongly with existing measure of same construct.
Divergent: tests should not correlate with a construct that is not theoretically related.

26
Q

What is Test Bias?

A

Systematically prevents accurate and impartial measurement.

27
Q

What are the Types of Measurement Bias?

A

Intercept bias: consistently over/under-predicting.
Slope bias: shows weaker correlation for specific groups compared to others.
Rating error: (un)intentional judgement error resulting from misuse of scale.

28
Q

What is Test Fairness?

A

Extent to which a test is used in an impartial. just and equitable way.

29
Q

What is a Nomological Net?

A

Describes what other constructs your construct is related to.

3006PSY (4 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions