F&B Validity

Question 1

What is meant by validity?

Answer 1

the degree to which evidence and theory support the interpretations of test scores entailed by the proposed uses

Question 2

What part of a test is seen as valid or invalid

Answer 2

The interpretation of the score and its implications; not any part of the test

Question 3

How is validity traditionally a three faceted concept?

Answer 3

Content validity- the degree to which test scores can be interpreted as as reflecting a particular psychological construct
Criterion validity - the degree to which test scores can predict specific criterion variables.
Construct validity

Question 4

What five types of evidence relevant for establishing the validity of test score interpretation was published in the Standards for Educational and Psychological Testing?

Answer 4

Internal structure 
Response processes
Test Content
Consequences of use
Associations with other variables

Question 5

Name 2 threats to content validity

Answer 5

Construct-irrelevant content

Construct underrepresentation

Question 6

What is meant by face validity?

Answer 6

the degree to which a measure appears to be related to a specific construct, in the judgment of non-experts, such as test takers and representatives of the legal system

Question 7

Why may test validity be relevant?

Answer 7

The apparent meaning and relevance of a lest’s content might influence lest takers’ motivation to respond in a serious and honest manner.

Question 8

What is meant by the internal structure of a test?

Answer 8

The way the parts of the test are related to each other (correlated items or clusters etc)

Question 9

What should the internal structure of a test match if is to be internally valid?

Answer 9

Internal structure of the theory

Question 10

What test examines the internal structure of a test?

Answer 10

Factor analysis

Question 11

How are the results of a factor analysis displayed for “global” and for multidimensional tests

Answer 11

Global- scree plot (eiganvalues)

Multidimensional- orthogonal rotation

Question 12

What is the facet “associations with other variables” concerned with?

Answer 12

The match between a measure’s actual associations with other variables and the associations is actually has with other variables

Question 13

When evaluating the pattern of validity associations between a measure and other measures, what evidence should we consider? (2)

Answer 13

Convergent evidence- the degree to which scores are correlated to tests of related (or the same) constructs
Discriminant evidence- the degree to which test scores are uncorrelated with tests of unrelated constructs

Question 14

Distinguish between concurrent validity evidence and predictive validity evidence

Answer 14

Concurrent validity evidence refers to the degree to which test scores are correlated with other relevant variables that are measured at the same time as the primary test of interest.
Predictive validity evidence refers to the degree to which test scores are correlated with relevant variables that are measured at a future point in time.

Question 15

concurrent validity evidence and predictive validity evidence are variants of what type of what type of evidence

Answer 15

Convergent evidence

Question 16

Name two big differences between the contemporary and traditional perspective on validity

Answer 16

-the contemporary perspective emphasizes the primacy of construct validity over content
validity and criterion validity
-The assertion from the contemporary perspective that the social consequences are a facet of validity

Question 17

According to the contemporary view, what should criterion validity be categorised under?

Answer 17

Construct validity

Question 18

How would an inductive view of testing validity look?

Answer 18

by examining the associations between test scores and a large set of potentially
important and relevant psychological variables rather than testing the hypothesis of what is is assumed to test

Question 19

When would an inductive method usually be utilised?

Answer 19

In a research context

Question 20

What is meant by the nomological network?

Answer 20

The interconnections between a construct and other related constructs

Question 21

Name four methods for evaluating convergent and discriminant validity

Answer 21

Focused associations
Sets of Correlations
Multitrait-Multimethod Matrices

Question 22

What is involved in focused associations?

Answer 22

When measures have clear relevance for very few very specific variables

Question 23

What are meant by validity coefficients?

Answer 23

Correlations between test scores and crucial variables

Question 24

What is meant by validity generalisation?

Answer 24

Process of evaluating a tests validity coefficients across a large set of studies, it is a form of meta-analysis- it studies a range of settings, times, situations etc

Question 25

What three issues does a validity generalisation study address?

Answer 25

1. Can reveal the general level of predictive validity across all of the smaller individual studies
2. Can reveal the degree of variability among the smaller individual studies (determining generalisability)
3. The source of variability among studies (confound/ moderate etc)

Question 26

When is a set of correlations utilised?

Answer 26

When a constructs nomological network touches on a wide variety of other constructs with varying levels of association rather than a small set of extremely relevant constructs

Question 27

How is a set of correlations utilised?

Answer 27

First researchers compute the correlations between a test of interest and measures of many criterion variables. They will then ‘eyeball’ the correlations and make a somewhat subjective judgement about the degree to which the correlations match.

Question 28

How is a mutitrait multi method study carried out?

Answer 28

Researchers obtain measures of several traits, each of which is measured through several methods (the test, self report, friends, psychologist etc)

Question 29

How does MTMMM study variance?

Answer 29

By both studying trait variance and method variance

Question 30

What is meant by shared method variance?

Answer 30

When there is a correlation because it is derived from two measures from the same source (bias in self report etc)

Question 31

How may method variance affect results?

Answer 31

A strong correlation could either mean the two traits have some commonalities (share trait variance) or that they share method variance. On the other hand a weak correlation could indicate very few commonalities or that they share no method variance

Question 32

Name the four different types of correlations in a MTMMM and their expected strengths

Answer 32

Heterotrait- heteromethod correlations- weakest
Heterotrait- mono method correlations- moderate
Monotrait- heteromethod correlations- moderate
Monotrait- mono method correlations- strongest

Question 33

What is convergent validity represented by in MTMMM?

Answer 33

Monotrait- heteromethod correlations

Question 34

What requirement is there for strong evidence for convergent validity?

Answer 34

The mono trait- heteromethod correlations should be higher than heterotrait- mono method measures or heterotrait- heteromethod measures