2-6 tb

Question 1

traditional views of validity

Answer 1

a test is valid if it measures what it is designed to measure

a test can measure exactly what it was designed to measure and yet not be valid if the inferences that are made from the test scores are not supportable

e.g. making a hiring decision based on a personality test

Question 2

a test is valid if it measures what it is designed to measure

a test can measure exactly what it was designed to measure and yet not be valid if the inferences that are made from the test scores are not supportable

e.g. making a hiring decision based on a personality test

Answer 2

traditional views of validity

Question 3

current views of validity

Answer 3

view validity as a unitary or single concept

importance of evaluating the interpretation of test scores and then agree to which the accumulated evidence supports the intended interpretation of test scores for the proposed use

Question 4

view validity as a unitary or single concept

importance of evaluating the interpretation of test scores and then agree to which the accumulated evidence supports the intended interpretation of test scores for the proposed use

Answer 4

current views of validity

Question 5

What are the five sources of evidence of validity currently recognized by the Standards?

Answer 5

1. evidence based on test content
2. evidence based on response processes
3. evidenced based on internal structure
4. evidence based on relations with other variables
5. evidence based on the consequences of testing

Question 6

1. evidence based on test content

Answer 6

Previously referred to as content validity, this source of validity evidence involves logically examining and evaluating the content of a test (including the test questions, format, wording, and tasks required of test takers) to determine the extent to which the content is representative of the concepts that the test is designed to measure without either underrepresenting those concepts or including elements that are irrelevant to their measurement.

Question 7

Previously referred to as content validity, this source of validity evidence involves logically examining and evaluating the content of a test (including the test questions, format, wording, and tasks required of test takers) to determine the extent to which the content is representative of the concepts that the test is designed to measure without either underrepresenting those concepts or including elements that are irrelevant to their measurement.

Answer 7

1. evidence based on test content

Question 8

2. evidence based on response processes

Answer 8

bserving test takers as they respond to the test and/or interviewing them when they complete the test

Question 9

bserving test takers as they respond to the test and/or interviewing them when they complete the test

Answer 9

2. evidence based on response processes

Question 10

3. evidenced based on internal structure

Answer 10

focuses on whether the conceptual framework used in test development could be demonstrated using appropriate analytical techniques

e.g. if a test was designed to measure a single concept (such as anxiety), we would analyze the test results to find out how many underlying concepts account for the variations in test taker scores.

Question 11

focuses on whether the conceptual framework used in test development could be demonstrated using appropriate analytical techniques

e.g. if a test was designed to measure a single concept (such as anxiety), we would analyze the test results to find out how many underlying concepts account for the variations in test taker scores.

Answer 11

3. evidenced based on internal structure

Question 12

4. evidence based on relations with other variables

Answer 12

criterion-related validity

correlating test scores with other measures to determine whether those scores are related to other measures to which we would expect them to relate

Question 13

criterion-related validity

correlating test scores with other measures to determine whether those scores are related to other measures to which we would expect them to relate

Answer 13

4. evidence based on relations with other variables

Question 14

5. evidence based on the consequences of testing

Answer 14

intended and unintended consequences may occur

if the test is biased, an unintended consequence might be that test scores appear to favor one group over another. However, it is also important to understand that just because different groups score differently on a test does not automatically mean that the test is biased

Question 15

intended and unintended consequences may occur

if the test is biased, an unintended consequence might be that test scores appear to favor one group over another. However, it is also important to understand that just because different groups score differently on a test does not automatically mean that the test is biased

Answer 15

5. evidence based on the consequences of testing

Question 16

What are the three main categories and strategies for gathering evidence of validity?

Answer 16

1. evidence based on test content/content validity
2. evidence based on relations with other variables/criterion-related validity
3. evidence based on relations with other variables/construct validity

Question 17

1. evidence based on test content/content validity

Answer 17

content validity: how well the test represents the material it’s covering

Question 18

how well the test represents the material it’s covering

Answer 18

1. evidence based on test content/content validity

def of content validity

Question 19

1. evidence based on test content/content validity

How do we know when test items show evidence of content validity?

Answer 19

observable and measurable behaviors

e.g. play an instrument

????

Question 20

2. evidence based on relations with other variables/criterion-related validity

What are the similarities and differences between predictive validity and concurrent validity?

Answer 20

Sim: both measure how well the test does to other measures (e.g. compare to similar test or how well it correlates with supervisor performance rating)

Diff: predictive how well they’ll do on a criterion measure (e.g. performance rating) at a later time vs concurrent measures how the scores are related to scores on other measurement that have established validity at about the same time

Question 21

2. evidence based on relations with other variables/criterion-related validity

What is predictive validity? Predictive method?

Answer 21

Predictive Validity: a test’s relationships with other variables that shows a relationship between test scores obtained at one point in time and a criterion measured at a later point in time (e.g. supervisor ratings)

Predictive Method: want to show a relationship between test scores and a future behavior

Question 22

2. evidence based on relations with other variables/criterion-related validity

What is concurrent validity? Concurrent method?

Answer 22

Concurrent Validity: a test’s relationships with other variables in which test administration and criterion measurement happen at roughly the same time

Concurrent Method: administering two measures, the test and a second measure of the attribute, to the same group of individuals at as close to the same point in time as possible (e.g. take an American Lit test and the second measure is the grade in the class)

Question 23

3. evidence based on relations with other variables/construct validity

What is meant by convergent evidence of validity? Example?

Answer 23

If the test is measuring a particular construct, we expect the scores on the test to correlate strongly with scores on other tests that measure the same or similar constructs.

example: researchers have developed a number of tests to measure general self-efficacy as well as self-efficacy related to a specific task. We would expect two measures of general self-efficacy to yield strong, positive, and statistically significant correlations.

Question 24

3. evidence based on relations with other variables/construct validity

What is meant by discriminant evidence of validity? Example?

Answer 24

When the test scores do not correlate with unrelated constructs, we can say that the test is demonstrating discriminant evidence of validity

example: a test that measures skill at performing numerical calculations would not be expected to correlate with a test that measures reading comprehension

Question 25

Appropriate use of various validation strategies

Answer 25

Evidence based on content (content validity): for tests that measure concrete attributes (observable and measurable behaviors—play an instrument)

Evidence based on relations with external criteria: validity (criterion): for tests that predict outcomes (job performance, success in college)

Evidence based on relations with other constructs (construct validity): for tests that measure abstract constructs (like anxiety, depression, our course project)

Question 26

evidence of validity based on content

2 of them

Answer 26

During test development

After test development

Question 27

during test development

Answer 27

evidence of validity based on content

the first method for obtaining evidence of validity based on the content of a test involves performing a series of systematic steps as a test is being developed

define the testing universe, develop the test specifications, establishing a test format, constructing test questions

Question 28

the first method for obtaining evidence of validity based on the content of a test involves performing a series of systematic steps as a test is being developed

Answer 28

during test development

evidence of validity based on content

define the testing universe, develop the test specifications, establishing a test format, constructing test questions

Question 29

After Test Development

Answer 29

evidence of validity based on content

experts rate how essential item are to the attribute being measures

content validity ratio is calculated, providing a measure of agreement among the judges.

Question 30

experts rate how essential item are to the attribute being measures

content validity ratio is calculated, providing a measure of agreement among the judges.

Answer 30

After Test Development

evidence of validity based on content

Question 31

evidence of validity based on test content: summary

Answer 31

When purchasing a test, users should not make any assumptions about whether the test can show evidence of validity based on its content

compare the specifications of the purchased test with the content domain of the test

Question 32

When purchasing a test, users should not make any assumptions about whether the test can show evidence of validity based on its content

compare the specifications of the purchased test with the content domain of the test

Answer 32

evidence of validity based on test content: summary

Question 33

face validity

Answer 33

the perception of the test taker that the test measures what it is supposed to measure

Question 34

the perception of the test taker that the test measures what it is supposed to measure

Answer 34

face validity