Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

psyc 406 > Midterm 1 - Validity (Ch. 5 and Miller et al.) > Flashcards

Midterm 1 - Validity (Ch. 5 and Miller et al.) Flashcards

1
Q

MTMM matrices are particularly useful bc they provide us with ______ coefficients, ____ coefficients, and _____ coefficients

A

convergent validity, discriminant validity, reliability

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

what is the difference btw convergent and discriminant validity evidence?

A

convergent: degree to which data from two measures targeting the same construct agree (or correlate)

discriminant: degree to which data from two measures targeting distinct constructs disagree (or do not correlate)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

In the MTMM, validity diagonals consist of validity coefficients depicting the correlations between _____ and _____

A

the same trait measured using distinct methods (monotrait-heteromethod values)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

In the MTMM, discriminant validity coefficients are organized into (hetero/mono)trait-(hetero/mono)method triangles which depict correlations among ________

A

heterotrait-monomethod and heterotrait-heteromethod triangles

HTMM: corr. btw measures of diff constructs using same method
HTHM: corr. btw measures of diff constructs using diff methods

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

In the MTMM, reliability coefficients are computed by ____

A

not calculated in the matrix, just placed where monotrait-monomethod coefficients would go to depict relative reliability of each measure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Validity assesses the extent to which a test _____

A

measures the attribute/construct it is designed to measure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Validity is not a yes/no decision. It comes in ___

A

degrees (no specific range of coefficients)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

generating validity evidence is ____

A

an ongoing process

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What are the 3 types of validity?

A
  • content validity
  • criterion validity
  • construct validity
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What is face validity?

A
  • 4th type of validity but not considered actual/sufficient form of validity
  • whether test appears to measure what it is supposed to measure
  • can help ppl trust the test and take it seriously BUT can also run into issues w responses based on social desirability
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Content validity evaluates adequacy of ____

A

domain representation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What are the 3 steps of establishing content validity?

A
  1. describe content domain (identify boundaries, determine structure-eg proportion of test on a given topic)
  2. inspect test
  3. form judgement that test measures what it is supposed to measure
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Content validity is high when the test has good _____ and when _____

A

good content coverage (representative of entire domain) AND relevant items (inside content domain)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Why is content validity not sufficient to decide a test is valid?

A

no info ab relation of test to external constructs or external variables!

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Criterion validity evaluates effectiveness of the test in _____

A

predicting narrowly and specifically identified outcomes/criteria

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What is a criterion? Is it objective or subjective?

A
  • standard researchers use to measure outcomes such as performance or attitude
  • can be objective (observable and measurable) or subjective (based on a person’s judgement)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

What are the 2 subtypes of criterion validity?

A
  • concurrent validity: criterion available at same time as test
  • predictive validity: criterion measure available in the future
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What are the differences between concurrent and predictive validity?

A
  • concurrent: large group takes test and another measure (criterion) AT SAME POINT IN TIME
  • predictive: large group takes test and scores are held, criterion measure is administered LATER
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What is the impact on validity if…
…a criterion measures fewer dimensions than those measured by the test?
…a criterion measures more dimensions than those measured by the test?

A
  • decreased evidence of validity
  • criterion contamination (also decreased validity)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What is a validity coefficient?

A

correlation btw test and criterion

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

validity coefficients are rarely greater than ____

A

.60

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

(T/F) correlation is a measure of effect size

A

TRUE

23
Q

What are 3 factors limiting validity coefficients?

A
  • restricted range of scores decreases validity coefficients
  • low reliability of test scores decreases validity coefficients
  • low reliability of criterion decreases validity coefficients
24
Q

What is the formula to correct for unreliability in test?

A

rxycorr = rxy / √rxx’

rxycorr = estimated validity coefficient
rxy = observed validity coefficient (of unreliable measure)
rxx’ = reliability coefficient of predictor test

25
Q

What is the formula to correct for unreliability in criterion?

A

rycorrxcorr = ryx / √(rxx’*ryy’)

rycorrxcorr = estimated validity coefficient
ryx = observed validity coefficient (of criterion)
rxx’ = reliability coefficient of predictor test
ryy’ = reliability coefficient of criterion

26
Q

What is an issue with the formula to correct for unreliability in criterion?

A

can sometimes generate unrealistic numbers (eg above 1)

27
Q

Why can small validity coefficients still have practical utility?

A
  • can use to compare validity between different types of tests
  • ex study that compared validity of psych vs medical tests
28
Q

The process of construct validation involves accumulating information indicating that _____

A

the test acts as it is supposed to act given the construct definition

29
Q

What are the 3 steps of construct explication? Together, these steps help to establish a _______

A
  1. identify behaviours that relate to construct
  2. identify other related constructs
  3. identify behaviours related to other constructs

nomological network!

30
Q

What are the 6 steps to gather psychometric evidence of construct validity?

A
  • evidence of validity based on content
  • evidence of validity based on relations w criteria
  • reliability
  • experimental interventions
  • convergent evidence of validity
  • discriminant evidence of validity
31
Q

The nomological network consists of ____, ____, and ____

A

constructs, their observable manifestations, and the relations within and between constructs and their observable manifestations

32
Q

In examining evidence of validity based on content, we want to make sure there is no _____ and no ____

A

no construct underrepresentation
no irrelevant construct representation

33
Q

In examining evidence of validity based on relations w criteria, we are looking to see if _____

A

the relations of the test w external criteria are as would be expected based on theory

34
Q

In examining evidence of validity based on reliability of the test, we need to make sure that ______ is not too (high/low)

A

test-retest or internal consistency is not too high or low given the construct

35
Q

In examining evidence of validity based on experimental interventions, we are checking to see that _____

A

situational changes that should influence the test scores DO influence test scores (eg starting anxiety meds should reduce score on anxiety measure)

36
Q

What is the difference between convergent and discriminant validity?

A
  • convergent: test scores correlate w other measures of same construct OR measures of constructs to which test should be related (think nomological net)
  • divergent: test scores UNcorrelated w measures of constructs to which the construct should not be related (think nomological net)
37
Q

method variance is a form of ___ error

A

systematic

38
Q

MTMM allows us to examine ___, ___, and ___

A

divergent validity, convergent validity, and method variance

39
Q

In using the MTMM, we are looking for:
- (high/low) trait variance (variance due to construct)
- (high/low) method variance
- (high/low) irrelevant variance (variance shared w unrelated measures)

A
  • HIGH trait variance
  • LOW method variance
  • LOW irrelevant variance
40
Q

What are the minimum requirements to run a MTMM?

A

at least 2 measures representing 2 diff methods for each of 2 constructs

41
Q

In the MTMM, monomethod blocks are located _____. They contain ______ and _____.

A

along the main diagonal
contain reliability diagonal (middle of table) and heterotrait-monomethod triangles

42
Q

What does the reliability diagonal tell us?

A
  • general reliability of test (eg test-retest,, internal)
  • can use these to compare relative reliability of each
43
Q

What do heterotrait-monomethod triangles tell us? Do we want these numbers to be high or low?

A
  • tell us if there is method variance
  • should be low (if high, means there is method variance)
  • generally should be lower than reliability diagonal
44
Q

In the MTMM, heteromethod blocks are located ____. They contain ____ and ____.

A

at bottom left of table (in an L)
contain validity diagonals and heterotrait-heteromethod triangles (lying on each side of validity diagonals)

45
Q

What do the validity diagonals tell us? Do we want these numbers to be high or low?

A
  • they are monotrait-heteromethod
  • tells us correlation of same trait measured w diff methods
  • we want this to be as high as possible (convergent validity)
46
Q

What do the heterotrait-heteromethod triangles tell us? DO we want these numbers to be high or low?

A
  • can use as evidence of divergent validity
  • want to be as low (close to 0) as possible!
47
Q

How can you assess which method has less method variance using MTMM?

A
  • look at heterotrait-monomethod triangles
  • avg each triangle (method)
  • lowest avg has least method variance
48
Q

Correlations btw diff constructs using same method should be (higher/lower) than convergent validity coefficients (validity diagonals)

A

LOWER

49
Q

How can you assess convergent validity of constructs using MTMM?

A
  • look @ validity diagonals
  • avg of 3 correlations for each construct
  • highest avg has best convergent validity
50
Q

What are 3 advantages of the MTMM approach?

A
  • allows examining convergent and discriminant validity simultaneously
  • stresses the characteristics that a good test should have
  • reflects nature of construct validation (can’t just use a single coefficient!)
51
Q

What is a disadvantage of the MTMM approach?

A
  • sometimes not feasible (high demands)
52
Q

(T/F) a test can be reliable without being valid

A

TRUE

53
Q
A

psyc 406 (16 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions