Test Validity - Content and Construct Validity

Question 1

Validity has been traditionally defined as the degree to which a test accurately measures what it was designed to measure, and a distinction has been made between three types of validity:

Answer 1

1. Content Validity
2. Construct validity
3. Criterion-related

Question 2

_____ is established during the development of a test by clearly defining the domain to be assessed and including items that are a representative sample of that domain. Test items are then systematically reviewed by subject matter experts to ensure the items address all important aspects of the domain.

Answer 2

Content validity

Question 3

Evidence of _____ is important for tests that have been designed to measure a hypothetical trait (e.g., intelligence, motivation, introversion) that cannot be directly observed but is inferred from an examinee’s behavior. It involves using several procedures including obtaining evidence of the test’s convergent and divergent validity

Answer 3

construct validity

Question 4

In terms of construct validity, _____ refers to the degree to which scores on the test have high correlations with scores on other measures designed to assess the same or related constructs.

Answer 4

Convergent validity

Question 5

In terms of construct validity, _____ refers to the degree to which scores on the test have low correlations with scores on measures of unrelated constructs.

Answer 5

Divergent Validity (also known as discriminant validity)

Question 6

Factor Analysis is used for several purposes including assessing a test’s convergent and divergent validity. It involves four basic steps:

Answer 6

(1) Administer the test that’s being validated to a sample of examinees along with tests of the same or related traits and unrelated traits.
(2) Correlate all pairs of scores and list the correlation coefficients in a correlation (R) matrix.
(3) Use the data in the correlation matrix to derive the initial factor matrix.
(4) Rotate the initial factor matrix and interpret and name the factors. (The factor matrix is rotated because the rotated matrix produces data that’s easier to interpret than the data in the initial matrix.) For the exam, you want to be familiar with the output of the final step – the rotated factor matrix.

Question 7

The _____ is a table of correlation coefficients that provide information about a test’s reliability and convergent and divergent validity. When using the matrix, the test to be validated is administered to a sample of examinees along with at least three other measures: a test of the same trait using a different method, a test of an unrelated trait using the same method, and a test of an unrelated trait using a different method.

Answer 7

multitrait-multimethod matrix

Question 8

In factor analysis, _____ are correlation coefficients that indicate the correlation between each test and each identified factor.

Answer 8

factor-loadings

Question 9

In factor analysis, the _____ column indicates the amount of variability in each test that’s explained by all of the identified factors.

Answer 9

communality

When the communality is not listed in the table, it can be calculated by squaring and adding the factor loadings as long as the factors are orthogonal (uncorrelated) but not when they’re oblique (correlated).