Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Psyc3302 > Week 5-6 Validity > Flashcards

Week 5-6 Validity Flashcards

1
Q

validity

A

the degree to which evidence and theory support the interpretations of test scores entailed by the proposed uses of a test

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

four important implications of this definition of validity

A

1 Validity concerns interpretations and uses of scores
2 Validity is not a property of the test itself
3 Validity is a matter of degree
4 Validity is based on theory and evidence

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Validity is a crucial basis for:

A

1 the meaningful interpretation of behavioural research
2 making sound societal decisions based on such research
3 making informed test-based decisions about individuals

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Content validity:

A

The degree to which the content of a test is representative of the domain it’s supposed to cover.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

criterion-related validity

A

a measure obtained by evaluating the relationship of scores obtained on the test with scores on other tests

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

construct validity

A

a measure obtained by performing an analysis of:

a) how scores on the test relate to other test scores and measures, and
b) how scores on the test can be understood within some theoretical framework

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Test Content

A

This is the match between the content of a test and the content that should be included in the test

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

two types of validity relevant to test content:

A

1 Content validity

2 Face validity

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

two key threats to content validity:

A

Construct-irrelevant content

Construct under-representation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Face validity

A

what a test appears to measure to the person being tested, rather than what the test actually measures

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Content validity can be evaluated only by ________ in a field, whereas face validity must be assessable by __________ •

A

Experts, non-experts

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Internal structure

A

the way the parts of a test are related to each other:
• some tests include items that are highly correlated with each other, forming a single cluster
• other tests include items that fall into two or more clusters

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Factorial validity concerns the match between the ________ internal structure of a test and the structure the test ________ possess

A

actual ,should

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Response processes

A

there should be a close match between the psychological processes that the respondents actually use when completing a measure, and the process that they should use

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

methods for obtaining validity evidence of the response processes include:

A
  • “think-aloud” procedures
  • cognitive interviews
  • focus groups
  • response times
  • eye movements
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Associations with other variables

A

the way in which the construct is connected to other relevant psychological variables

  • Our theoretical understanding of the construct we are trying to measure should lead us to expect a particular pattern of associations with other variables
  • This type of validity evidence emphasises the match between measures predicted and observed associations with other measures
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Convergent evidence (convergent validity)

A

the degree to which test scores are correlated with tests of related constructs

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Discriminant evidence (discriminant validity)

A

the degree to which test scores are uncorrelated with tests of unrelated constructs

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Concurrent validity evidence

A

the degree to which test scores are correlated with other relevant variables that are measured at the same time as the test undergoing validation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

Predictive validity evidence

A

the degree to which scores on the test undergoing validation are correlated with relevant variables that are measured at a future point in time

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

consequential validity

A

refers to the social and personal consequences associated with using a particular test

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

what kind of test is associated with greater consequential validity

A

a non-biased test

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

three other types of validity that arguably do not fit as strongly within this construct/theory framework:

A

1 Criterion Validity
2 Induction-Construct Development Interplay
3 Measurement as Theory

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

A criterion

A

the standard against which a test or test score is evaluated

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

2 examples of criterion validity

A

Concurrent validity and predictive validity

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

Induction-Construct Development Interplay

A

bottom up theory development, e.g FFM

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
27
Q

Measurement as theory

A

This approach rejects much of the unitary view except the importance attached to constructs and the theoretically based examination of response processes

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
28
Q

Internal consistency reliability is typically estimated with

A

Cronbach’s α

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
29
Q

Cronbach’s α assumes

a) unidimensionality
b) multidimensionality

A

unidimensionality

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
30
Q

Factorial validity concerns the ___________ ___________ of test scores

A

internal structure

31
Q

The test items on a unidimensional test would also have the property of ___________ ____________

A

conceptual homogeneity

32
Q

Is a total test score calculated for a multidimensional tests with uncorrelated dimensions?

A

no, a score is obtained for each dimension, but the dimensions scores are not combined to compute a total test score

33
Q

The communality for a given variable can be interpreted as

A

the percentage of variation in that variable explained by the extracted components

34
Q

As a general statement, you want to see communalities that are at least

A

.04 for items or .09 for subscales

35
Q

Items are less reliable than subscales, so they have _______ communality expectations

A

lower

36
Q

The sum of the “Initial Eigenvalues” is equal the number of

A

variables included in the analysis

37
Q

Component loadings refer to the associations between ______ and ____________

A

items, components

38
Q

Component loadings vary between

A

-1 and +1

39
Q

The size of the loading indicates the degree of association between a _______ and a ______________

A

item, component

40
Q

Component loadings further from 0 indicate ____________ associations

A

stronger

41
Q

A positive loading indicates that people who respond with a high score on an item have a ________ level of the underlying component

A

high

42
Q

A negative loading indicates that people who respond with a high score on an item have a _____ level of the underlying component

A

low

43
Q

As a general statement, useful component loadings are either _____ or ____ or greater

A

.20 (Items) or .30 (subscales)

44
Q

Simple structure occurs when each item is strongly linked to

A

one and only one component

45
Q

One way to help achieve simple structure is to

A

rotate the solution

46
Q

should you always rotate the solution?

A

Yes

47
Q

How many components do you need to rotate the solution?

A

two or more

48
Q

If you extract only one component, then there will only be the

A

“Component Matrix”

49
Q

If you have a minimum of 5 variables per factor and the communalities all exceed _____, then a sample size of about should be sufficient

A

.20, 150

50
Q

There are no simple guidelines to follow, because the size of the sample required will be dependent upon two main factors:

A
  • the amount of communality associated with the variables (higher communality means less sample size required)
  • the number of variables per factor (higher number of variables per factor means less sample size required)
51
Q

4 Methods of evaluating construct validity

A

1 Focussed associations
2 Sets of correlations
3 Multitrait-Multimethod Matrices
4 Quantifying Construct Validity

52
Q

3 Factors affecting validity coefficients

A

1 True associations
2 Measurement error
3 Restricted range

53
Q

The interconnections between a construct and other related constructs are known collectively as a

A

nomological network

54
Q

validity coefficient

A

correlation coefficient between a test score (predictor) and a performance measure (criterion)

55
Q

Example of a focused association

A

SAT and first year university marks

56
Q

Validity generalization studies are intended to evaluate the predictive utility of a test’s scores across

A

a range of settings, times, situations, etc

57
Q

Sets of correlations

A

large nomological networks incorporating a wider variety of other constructs, with differing levels of association with the main construct

58
Q

validity generalization

A

is a process of evaluating a test’s validity coefficients across a large set of studies, comparable to a meta analysis

59
Q

Is the judgement about the degree to which the pattern of coefficients matches the expectation

a) objective
b) subjective

A

A subjective

60
Q

Validity generalization studies can essentially address three questions:

A

1 estimate the average level of predictive validity across studies
2 estimate the degree of variability associated with the validity coefficients
3 identify sources of systematic variability in the validity coefficients

61
Q

a correlation between two scores may conflate two sources of variance

A
  • trait variance (the good stuff)

* method variance (the bad stuff)

62
Q

large correlations between different traits using the same measurement method suggests that the correlations are simply due to

A

a response style (i.e., method variance)

63
Q

using MTMMM we want a lot shared trait variance, particularly identical traits measured using

A

different methods

64
Q

The multitrait component of the study refers to

A

administer the questionnaire measuring the trait of interest in addition to other measures such as a measure of impulsivity, conscientiousness, and emotional stability

65
Q

the multimethod component refers to

A

measuring the trait of interest and the additionally trait using multiple methods, e.g self report and acquaintance report

66
Q

Sources of variance for Heterotrait-Heteromethod correlations

A

Nonshared trait variance, nonshared method variance

67
Q

The most stringent test in a MTMM analysis is to determine whether the ________________ _________________- correlations are “meaningfully” larger than the ____________- ________________ correlations

A

monotrait-heteromethod, heterotrait-monomethod

68
Q

one of the limitations associated with MTMMM

A

There are no clear guidelines to evaluate the differences in the mean correlations

69
Q

At the very least, the ___________ ___________ correlations need to be larger than the ______________ _____________correlations

A

monotrait-heteromethod,heterotrait-monomethod

70
Q

Quantifying construct validity (QCV)

A

this procedure requires researchers to predict the magnitude of the correlation between their measure of interest and their selected criteria

71
Q

3 advantages of the QCV approach

A

1 It forces researchers to consider carefully the expected pattern of convergent and discriminant associations that would make theoretical sense
2 It forces researchers to make explicit quantitative predictions about the pattern of associations
3 It provides a single value reflecting the overall “goodness-of-fit” between the predicted and actual associations

72
Q

using the QVC, why would a low correlation between the predicted and actual associations not necessarily reflect poor validity

A

because the predicted associations may simply be a poor reflection of a construct’s nomological network

73
Q

If a test’s or a criterion’s reliability is much lower than .70, there are two options:

A

1 disregard—or reduce the weight given to—a validity coefficient based on poor reliability
2 adjust the validity coefficient to account for measurement error

74
Q

a correlation between two variables can be reduced if the range of scores in one or both variables is

A

artificially limited or restricted

Psyc3302 (3 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions