Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

TOS PsychAssessment > Module 3: Validity > Flashcards

Module 3: Validity Flashcards

1
Q

Validity

A

+ a judgment or estimate of how well a test measures what it supposed to measure
+ evidence about the appropriateness of inferences drawn from test scores
+ degree to which the measurement procedure measures the variables to measure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Inferences

A

logical result or deduction

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Why may validity diminish?

A

May diminish as the culture or times change

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What is true about validity?

A

✓ Predicts future performance
✓ Measures appropriate domain
✓ Measures appropriate characteristics

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Validation

A

the process of gathering and evaluating evidence about validity

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Validation Studies

A

yield insights regarding a particular population of testtakers as compared to the norming sample described in a test manual

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Internal Validity

A

degree of control among variables in the study (increased through random assignment)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

External Validity

A

generalizability of the research results (increased through random selection)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Conceptual Validity

A

+ focuses on individual with their unique histories and behaviors
+ means of evaluating and integrating test data so that the clinician’s conclusions make accurate statements about the examinee

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Face Validity

A

a test appears to measure to the person being tested than to what the test actually
measures

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Types of Validity

A
  1. Content Validity
  2. Criterion Validity
  3. Construct Validity (Umbrella Validity)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Content Validity

A

+ describes a judgement of how adequately a test samples behavior representative of the universe of behavior that the test was designed to sample
+ representativeness and relevance of the assessment instrument to the construct being measured
+ when the proportion of the material covered by the test approximates the proportion of material covered in the course
+ more logical than statistical

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Test Blueprint

Content Validity

A

a plan regarding the types of information to be covered by the items, the no. of items tapping each area of coverage, the organization of the items, and so forth

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What is content validity concerned with?

A

concerned with the extent to which the test is representative of defined body of content consisting the topics and processes

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

How do a panel of experts evaluate a test for content validity?

A

panel of experts can review the test items and rate them in terms of how closely they match the objective or domain specification

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What does the content validity examine?

A

examine if items are essential, useful and necessary

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Construct underrepresentation:

A

failure to capture important components of a construct

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Construct-irrelevant variance

A

happens when scores are influenced by factors irrelevant to the construct

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Lawshe

A

developed the formula of Content Validity
Ratio

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

Formula of Content Validity Ratio

A

CVR=(Ne - N/2)/(N/2)

+ Ne is the number of panelists indicating “essential” and N is the total number of panelists

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

What is recommended when the CVI is low?

A

If the CVI is low, it is recommended to remove or modify the items that have low CVR values to improve the overall content validity of the test

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Zero CVR

A

exactly half of the experts rate the item as essential

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

Criterion Validity

A

+ more statistical than logical
+ a judgement of how adequately a test score can be used to infer an individual’s most probable standing on some measure of interest – the measure of interest
being criterion

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

Criterion

A

standard on which a judgement or decision may be made

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

Characteristics

A

relevant, valid, uncontaminated

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

Criterion Contamination

A

occurs when the criterion measure includes aspects of performance that are not part of the job or when the measure is affected by “construct-irrelevant” (Messick, 1989) factors that are not part of the criterion construct

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
27
Q

Types of Criterion Validity

A
  1. Concurrent Validity
  2. Predictive Validity
28
Q

Concurrent Validity

A

If the test scores obtained at about the same time as the criterion measures are obtained; economically efficient

29
Q

Predictive Validity

A

measures of the relationship between test scores and a criterion measure obtained
at a future time

30
Q

Incremental Validity

A

the degree to which an additional predictor explains something about the criterion measure that is not explained by predictors already in use; used to improve the domain

Type of Predictive Validity

31
Q

Construct Validity (Umbrella Validity)

A

+ covers all types of validity
+ logical and statistical
+ judgement about the appropriateness of inferences drawn from test scores regarding individual standing on variable called construct
+ test is homogenous
+ test score increases or decreases as a function of age, passage of time, or experimental manipulation
+ pretest-posttest differences
+ scores differ from groups
+ scores correlated with scores on other test in accordance to what is predicted

32
Q

Construct

A

+ an informed, scientific idea developed or hypothesized to describe or explain behavior; unobservable, presupposed traits that may invoke to describe test behavior or criterion performance
+ some constructs lend themselves more readily than others to predictions of change over time

33
Q

How can a test developer improve homogeneity of a test?

A

One way a test developer can improve the
homogeneity of a test containing dichotomous items is by eliminating items that do not show significant correlation coefficients with total test scores

34
Q

What makes a bad item in an academic test?

A

If it is an academic test and high scorers on the entire test for some reason tended to get that particular item wrong while low scorers got it right, then the item is obviously not a good one

35
Q

Method of Contrasted Groups

A

demonstrate that scores on the test vary in a predictable way as a function of membership in a group

36
Q

What will happen if a group takes a test with a construct that they do not have?

A

If a test is a valid measure of a particular construct, then the scores from the group of people who does not have that construct would have different test scores than those who really possesses that construct

37
Q

Convergent Evidence

A

if scores on the test undergoing construct validation tend to highly correlated with another established, validated test that measures the same construct

38
Q

Discriminant Evidence

A

a validity coefficient showing little relationship between test scores and/or other variables with which scores on the test being construct-validated should not be correlated

39
Q

Sensitivity

A

percentage of true positives

40
Q

Specificity

A

percentage of true negatives

41
Q

Multitrait-multimethod Matrix

A

+ useful for examining both convergent and discriminant validity evidence
+ the matrix or table that results from correlating variables within and between methods

42
Q

Multitrait

A

two or more traits

43
Q

Multimethod

A

two or more methods

44
Q

Factor Analysis

A

+ designed to identify factors or specific variables that are typically attributes,
characteristics, or dimensions on which people may differ
+ employed as data reduction method

45
Q

Who developed the Factor Analysis?

A

Charles Spearman

46
Q

What is the factor analysis used for?

A

+ used to study the interrelationships among set of variables
+ can be used to obtain both convergent and discriminant validity

47
Q

What does the factor analysis identify?

A

Identify the factor or factors in common between test scores on subscales within a particular test

48
Q

Types of Factor Analysis

A
  1. Explanatory FA
  2. Confirmatory FA
  3. Factor Loading
49
Q

Explanatory FA

A

estimating or extracting factors; deciding how many factors must be retained

50
Q

Confirmatory FA

A

researchers test the degree to which a hypothetical model fits the actual data

51
Q

Factor Loading

A

conveys info about the extent to which the factor determines the test score or scores

52
Q

Cross-Validation

A

revalidation of the test to a criterion based on another group different from the
original group form which the test was validated

53
Q

Types of Cross-Validation

A
  1. Validity Shrinkage
  2. Co-Validation
  3. Co-Norming
54
Q

Validity Shrinkage

Cross-Validation

A

decrease in validity after cross-validation

55
Q

Co-Validation

Cross-Validation

A

validation of more than one test from the same group

56
Q

Co-Norming

Cross-Validation

A

norming more than one test from the same group

57
Q

Bias

A

+ factor inherent in a test that systematically prevents accurate, impartial measurement
+ prejudice, preferential treatment
+ prevention during test dev through a procedure called Estimated True Score Transformation

58
Q

Rating

A

numerical or verbal judgement that places a person or an attribute along a continuum identified by a scale of numerical or word descriptors known as Rating Scale

59
Q

Rating Error

A

intentional or unintentional misuse of the scale

60
Q

Leniency Error

A

rater is lenient in scoring (Generosity Error)

61
Q

Severity Error

A

rater is strict in scoring

62
Q

Central Tendency Error

A

rater’s rating would tend to cluster in the middle of the rating scale

63
Q

What is one way to overcome rating errors?

A

One way to overcome rating errors is to use rankings

64
Q

Halo Effect

A

tendency to give high score due to failure to discriminate among conceptually distinct and potentially independent aspects of a ratee’s behavior

65
Q

Fairness

A

the extent to which a test is used in an impartial, just, and equitable way

66
Q

Would it be good to define the validity of a test if it is not reliable?

A

Attempting to define the validity of the test will be futile if the test is NOT reliable

TOS PsychAssessment (15 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions