Validity

Question 1

What is validity?

Answer 1

The degree to which evidence and theory support the interpretations of test scores for the proposed uses of a test.

Question 2

Three faceted concept

Answer 2

content validity, criterion validity, construct validity

Question 3

Content validity depends on

Answer 3

• content of the test
• internal structure of the test
• psychological processes used in test responses

Question 4

Importance of validity

Answer 4

• interpretation of behavioural research relies on test validity
• description accuracy depends on ability to measure/manipulate

Question 5

Validity evidence - test content -

Answer 5

Match between actual content of test and content that should be included

Question 6

Threats to content validity

Answer 6

• a test includes construct irrelevant content

- construct underrepresentation

Question 7

Internal structure of the test

Answer 7

The way in which parts of the test are related to each other

Question 8

Factor-analysis evidence

Answer 8

statistical procedure for evaluating internal structure

Question 9

response processes

Answer 9

match between psychological processes respondents actually use and should use

Question 10

direct evidence

Answer 10

obtained via interviews with respondents, discuss variety of issues

Question 11

Indirect evidence

Answer 11

eye tracking, statistical analysis of responses

Question 12

Convergent evidence

Answer 12

degree to which test scores correlate with measures of related constructs, to what degree are test scores correlated with other measures they should be?

Question 13

Discriminant evidence

Answer 13

degree to which test scores are uncorrelated with other measures they should be correlated with

Question 14

Concurrent evidence

Answer 14

degree test scores are correlated with relevant variables measured at the same time

Question 15

Predictive evidence

Answer 15

degree test scores correlated with relevant variables at future point in time

Question 16

4 methods to evaluate convergent and discriminate associations

Answer 16

• some more conceptually complex than others
• some more statistically complex
• some are old, some are new
• some require more explicit predictors than others

Question 17

Validity generalisation

Answer 17

evaluating validity coefficients across set of studies

Question 18

Validity generalisation can address important issues such as -

Answer 18

• reveal general level of predictive validity across smaller studies
• reveal degree of variability among smaller studies
• sources of variability among studies

Question 19

Factors affecting a validity coefficient

Answer 19

• associations between constructs
• random measurement error
• restricted large
• skew and relative proportions
• method variance
• time

Question 20

Interpreting a validity coefficient

Answer 20

Must decide whether coefficient is large enough to provide compelling evidence of convergent validity, small enough to provide assurance of discriminant validity

Question 21

Larger a correlation =

Answer 21

more successful making decisions

Question 22

Statistical significance

Answer 22

when statistically significant, researchers are confident that sample results are representative of population

Question 23

When evaluating convergent validity evidence -

Answer 23

expect to find statistically significant coefficients

Question 24

when evaluating discriminant validity evidence -

Answer 24

expect to find nonsignificant validity coefficients

Question 25

Facets

Answer 25

sources of variations and potential sources of error

Question 26

purpose

Answer 26

quantity amount of error caused by each facet on their interactions

Question 27

CIT vs G theory

Answer 27

• CIT only allows estimate of one type of error at a time

- G theory allows for variability in assessment conditions that may affect measurements

Question 28

consistency outcomes may change if measure used makes -

Answer 28

• absolute decisions

- relative decisions