Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Psychometrics > Chapter 6 > Flashcards

Chapter 6 Flashcards

1
Q

What is validity in psychological measurement?

A

The meaningfulness of a test score. What the test score actually means.

How well a test captures what it purports to capture

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Inference

A

A logical result or deduction

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Is there such thing as a universally valid test?

A

No. There are boundaries

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Local validation studies

A

Test users make their own validation studies for their purpose (eg. altering the test, different population, etc.)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Trinitarian view of validity

A

Content, Criterion-related, and Construct Validity

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Ecological Validity

A

How generalizable things are to real world

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Face validity

A

What a test appears to measure to the person being tested

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Content Validity

A

How well a test samples behavior representative of the universe of behavior that it’s designed to sample

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

How to ensure content validity in educational assessments?

A

Make sure test approximates proportion of material covered in the course

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Test blueprint

A

A plan for what information should be covered by items, how much of each area, organization on test, etc.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

How might (content) validity be relative?

A

Different cultures/religions/political parties have different historical interpretations. This means a test can be valid in one but not another region

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Criterion-Related Validity

A

How well a test score infers most probable standing on some measure of interest

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Concurrent validity

A

How much test score is related to some measure at the same time (concurrently)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Predictive validity

A

How well a test score predicts some criterion measure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Criterion (definition in testing evaluation)

A

The standard to which a test or score is being evaluated

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

An adequate criterion must be:

A

1) also Valid

2) Uncontaminated (a criterion measure cannot be based on its predictor)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

How do you statistically find criterion contamination or correct it?

A

Can’t

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Base rate vs. hit rate vs. miss rate

A

base rate: proportion that actually has something

hit rate: proportion accurately identified by a test to have something

miss rate: proportion that a test fails to identify to have something

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

2 statistics used for criterion-related validity

A

1) validity coefficient:
- relationship between test scores and scores on the criterion measure (eg. Pearson, Spearman)

2) expectancy data:

20
Q

What happens to validity coefficient when attrition of subjects happens over time?

A

Lower validity? (adverse effect)

21
Q

Incremental validity

A

How much an additional predictor explains something about criterion measure that isn’t already explained by the existing predictors

22
Q

Time spent studying (1) and time in library (2) likely has high or low incremental validity?

A

Low. Because they’re probably highly correlated

23
Q

Incremental validity of emotional intelligence

A

Has been doubted because of how they correlate strongly with intelligence and personality

24
Q

Construct Validity

A

how appropriate inferences drawn from a test are

25
Q

Construct (definition)

A

scientific idea that describes/explains behavior. unobservable/underlying

26
Q

Homogeneity of a test

A

Whether a test measures a specific concept

27
Q

If a concept meaning says the score is supposed to improve with age (eg. reading ability), measuring scores across ages would be evidence for what type of validity?

A

construct validity

28
Q

Pre-test and post-test of marriage thing after marriage therapy is evidence for which validity?

A

construct validity

29
Q

Method of contrasted groups

A

Show that scores differ between groups (one that should score higher vs. shoudn’t)

30
Q

Convergent evidence

A

eg. new anxiety test correlate with old

31
Q

Divergent evidence

A

evidence that something doesn’t correlate with something that it isn’t supposed to correlate with

32
Q

multitrait-multimethod matrix

what is it used for?

A

A matrix/table resulted from correlating traits within/between methods

used for convergent/divergent validity evidence

33
Q

Method variance

A

Similarity in scores due to the same method

34
Q

Factor analysis

A

Procedure to identify factors of attributes/characters/dimensions

35
Q

Factor loading

high or low for convergent/discriminant validity?

A

Determines extent to which a factor determines the test score.

high = convergent

36
Q

Intercept bias vs. Slope bias

A

intercept bias: consistent under/overprediction of a score of specific group

slope bias: a predictor has a weaker correlation with an outcome of specific groups

37
Q

Rating error

A

Misuse of a rating scale (eg. leniency error = grading too easy)

38
Q

Severity error

A

too harsh (opposite of leniency error)

39
Q

Central tendency error

A

Rater tends to grade average rather than positive/negative

40
Q

What might be a way to overcome rating errors?

A

Use ranking rather than scores. Forces a rank between individuals

41
Q

Fairness in psychometric context

A

How equitable, just, impartial a test is

42
Q

Does no members of a specific group being part of test make it unvalid for a test user in the group?

A

Not in itself.

43
Q

According to Borsboom article, what is validity? (when is a test valid?)

A

When the attribute exists and if variations in the attribute causes variations in outcomes

44
Q

Ontology vs. Epistemology (Borsboom)

A

Ontology: Existence of attributes

Epistemology: Process of knowing or finding out about the attributes

45
Q

Importance of causality in validity (Borsboom)

A

Validity is tied to causal relationships (changes in attribute –> changes in test score).

Currently, correlational views of validity are misconceptions. Tests aren’t valid because of correlations.

46
Q

What does Borsboom propose in test construction research?

A

To focus on causal processes between attributes and test outcomes rather than optimizing predictive properties (multicollinearity issues)

47
Q

How does correlational approaches not provide good evidence for validity?

A

They can lead to stupid conclusions, like thunder and lightning perfectly correlating, but you can’t say they’re the same thing.

Psychometrics (15 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions