2-7 tb

Question 1

What is evidence of validity based on test-criteria relationships?

Answer 1

?????

Question 2

criterion

Answer 2

the measure of performance (independent behaviors, attitudes, events) that we correlate with test scores

the extent to which a measure is related to an outcome.

Question 3

the measure of performance (independent behaviors, attitudes, events) that we correlate with test scores

the extent to which a measure is related to an outcome.

Answer 3

criterion

Question 4

Examples of criterion measures in clinical setting? Educational setting?

Answer 4

Clinical: psychologists often use tests to diagnose mental disorders

Educational: Educators use admissions tests to forecast how successful an applicant will be in college or graduate school (GRE)

Question 5

the predictive method

Answer 5

used when it is important to show a relationship between test scores and a future behavior

validity coefficient: the resulting correlation coefficient when two sets of scores are correlated–usually test scores (IV) and criterion score (DV)

Question 6

used when it is important to show a relationship between test scores and a future behavior

Answer 6

the predictive method

validity coefficient: the resulting correlation coefficient when two sets of scores are correlated–usually test scores (IV) and criterion score (DV)

Question 7

the concurrent method

Answer 7

when test administration and criterion measurement happen at the same time

Question 8

when test administration and criterion measurement happen at the same time

Answer 8

the concurrent method

Question 9

how does validity differ from reliability?

Answer 9

reliability is the consistency of the test

validity is the inferences based on test scores

test can be reliable but not valid (e.g. yardsticks are reliable for measuring distance, but not valid for measuring IQ)

Question 10

what are the two types of criterion

Answer 10

objective and subjective

Question 11

objective criterion

Answer 11

it is observable and measurable

e.g. number of accidents on the job

Question 12

it is observable and measurable

Answer 12

objective criterion

Question 13

subjective criterion

Answer 13

it is based on a person’s judgment

e.g. supervisor ratings

Question 14

it is based on a person’s judgment

Answer 14

subjective criterion

Question 15

Does the criterion measure what it is supposed to measure?

Answer 15

the criteria must be representative of events they are supposed to measure (content validity)

criteria measures fewer than what measured by test, it lacks evidence of content based validity

criteria measures more than the test, there is criterion contamination

Question 16

tests of significance

Answer 16

The process of determining what the probability is that a study would have yielded the observed results simply by chance

“How likely is it that the correlation between the test and the criterion resulted from chance or sampling error?”

Question 17

The process of determining what the probability is that a study would have yielded the observed results simply by chance

Answer 17

tests of significance

Question 18

p-value

Answer 18

the probability of obtaining test results at least as extreme as the results actually observed during the test, assuming that the null hypothesis is correct

Question 19

the probability of obtaining test results at least as extreme as the results actually observed during the test, assuming that the null hypothesis is correct

Answer 19

p-value

Question 20

p < .05

Answer 20

means the likelihood a relationship was found by chance or as a result of sampling error was less than 5 chances out of 100

Question 21

p < .01

Answer 21

means the likelihood a relationship was found by chance or as a result of sampling error was less than 1 chances out of 100

Question 22

coefficient of determination

Answer 22

answers the question “what amount of variance do the test and criterion share?”

The amount of variance shared by two variables being correlated, such as a test and a criterion, obtained by squaring the validity coefficient.

Question 23

The amount of variance shared by two variables being correlated, such as a test and a criterion, obtained by squaring the validity coefficient.

Answer 23

coefficient of determination

Question 24

What does the coefficient of determination tell us about the test and its relationship with a criterion?

Answer 24

For example, if the correlation (r) between a test and a criterion is .30, the coefficient of determination (r2) is .09. This means that the test and the criterion have 9% of their variance in common.

Question 25

How confident can we be about estimates of validity?

Answer 25

as long as the test is in use, need to keep validating it

use cross-validations and meta-analyses

Question 26

Using validity information to make predictions: Linear Regression

Answer 26

statistical procedure for predicting performance on a criterion using one set of test scores

DV is often the criterion

intercept is the predictor variable and the constant

Question 27

statistical procedure for predicting performance on a criterion using one set of test scores

Answer 27

Using validity information to make predictions: Linear Regression

Question 28

Using validity information to make predictions: Multiple Regression

Answer 28

statistical procedure for predicting a criterion using more than one set of test scores

equation includes multiple terms for the predictor variables and the constant (intercept)

DV is often the criterion

Question 29

statistical procedure for predicting a criterion using more than one set of test scores

Answer 29

Using validity information to make predictions: Multiple Regression

Question 30

Ethical issues associated with test validation

Answer 30

Test Users: be aware of test bias, make sure it’s valid for minorities, no questions specific to cultural backgrounds, appropriate norm groups for minorities

Test Publishers: make test manuals and information available/accessible before purchase and refuse to give test material to people not qualified to administer it