Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

PTPR > Validity > Flashcards

Validity Flashcards

Ch. 8 & 9

1
Q

What is the casual definition of validity?

A

The degree to which a psychological test measures what it purports to measure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What is the APA definition of validity?

A

The degree to which evidence and theory support the interpretation of test scores for proposed uses

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What are the three key points/differences between definitions?

A
  1. Validity is about the interpretation of test scores in terms of a specific psychological construct, it’s not about the test itself.
  2. Validity is a matter of degree; it’s not an “all-or-none” issue
  3. Validity is based on solid empirical evidence and theory
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Why is validity important?

A

It influences the scientific process, and in an abstract sense, it affects the accuracy of our understanding of the world.
It influences the decisions we make

Look at image 8.1 to see a diagram about this, not that important.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What is construct validity?

A

The degree to which test scores can be interpreted as reflecting a particular psychological construct

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What are the 5 facets of construct validity?

A
  1. Test content/content validity
  2. Response processes/response validity
  3. Internal structure/structural validity
  4. Associations with other variables/associative validity
  5. Consequences of use/consequential validity
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What is content validity?

I’ll go through each facet one at a time

A

The degree to which the actual content of a test matches the content that should be included in the test

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What evidence is content validity based on?

A

Its based on expert judgement of evidence, which typically involves different methods.
Such as content validity ratio, which represents the degree to which the experts see an item as essential to the test. (not that important)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What are the two threats to content validity?

A
  • Construct under-representation
    • Test fails to include content representing the full range of the construct
  • Construct irrelevant content
    • If the test includes content irrelevant to the construct
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What is face validity?

A

It is commonly confused with content validity, and it is validity in the eyes of the test user

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

How can face validity indirectly affect construct validity

A

If the test user doesn’t feel like the test is measuring what it’s supposed to (e.g. intelligence), they’ll be less motivated to respond correctly

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Response process validity until flashcard 15

What is response process validity?

A

The degree to which the psychological processes that respondents actually use when completing a measure match the processes that they should use.

E.g. using memory retrieval for a self-report question.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What direct evidence is response process validity based on?

A

Think out loud protocols (Getting the respondent to… think out loud)
Interviewing the respondent.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What indirect evidence is response process validity based on?

A
  • Process data (response times, mouse movements, eye movements)
  • Statistical analysis of the responses
  • Experimentally manipulating the response process
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What are some general threats to response process validity?

A

Respondents may use the wrong response process…

  • Due to poorly designed items
    • Misinterpretation, unintended correct solution, multiple correct solutions
  • Due to respondents
    • Guessing, social desirability, lack of motivation
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Internal structure validity until flashcard 18

What is internal structure validity?

A

The degree to which the actual internal structure of a test matches the structure that the test should possess
(E.g. A single theoretical construct having a unidimensional structure)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

What evidence is internal structure validity based on?

A

Factor analysis of test responses, which addresses fundamental issues for internal structure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What are the 3 issues in internal structure?

A
  1. Number of factors - They should match the number of dimensions within the construct(s).
  2. Associations among factors - once again, they should match the associations among the dimensions of the construct(s).
  3. Meaning of each factor (revealed by rotated factor loadings) should match the meaning of the dimensions of the construct(s).
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Associations among other variables validity until flashcard 45 (im sorry

What is associations among other variables validity?

A

The degree to which the test’s actual associations with other measures matches with the associations that the test should have with other measures.

View image 8.2 for an example

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What allows for an easier understanding of a constructs associations?
What does it linguistically mean?

A

A nomological network.
The network of “meaning” surrounding a construct.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

What is a nomological network?

A

It is a network that summarizes all theoretical relations between the construct of interest and other constructs and variables (such as behaviours or properties)
It dictates a particular pattern of associations/correlations among measures of those constructs.

See image 8.3 for an example of a nomological network for an intelligence test

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Why would depression be in a nomological network for intelligence?

A

Because in theory, they should not be correlated.
Therefore, if they are related, your measure is doing something wrong.
It counts as discriminant evidence

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

What is convergent validity?

A

When test scores are actually correlated with other measures that they should be correlated with

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

What is discriminant validity?

A

Test scores are actually uncorrelated (or weakly correlated) with other measures that they should not be correlated with

25
Q

Look at image 8.4, What level of validity does this show?
Why?

A

It shows moderate validity, because two relations comply, but one doesn’t.
* The correlations with self-awareness and critical thinking are good, they are what we expect as convergent evidence
* But since there is a negative correlation with depression, it lowers the validity. It should be as close to zero as possible since its discriminant evidence

You can now look at image 8.5 to see an example of good validity, as all observed relations comply to the theoretical relations.

26
Q

What is a validity coefficient?

We will now go over some terms related to analysing nomological networks

A

The individual correlation

27
Q

What is Validity generalization?

A

Establishing whether this pattern of correlations holds also for other tests (Could also be psychologist report)
For example
- Evaluating a test’s validity coefficients across a large set of studies
- It’s a form of meta analysis essentially

28
Q

What three issues does a validity generalization answer?

A
  • What is the average degree of validity across all studies
  • How much does validity vary from one study to another
  • Why does validity vary from one study to another
29
Q

What is criterion validity?

A

The association between the construct and an particularly important observed variable it should be theoretically related to

It’s typically analysed if an observed variable is including in a nomological network

30
Q

What is concurrent validity?

A

A subtype of criterion validity
The association between the construct and an observed variable measured at the same time

31
Q

What is predictive validity?

A

A subtype of criterion validity
The association between the construct and an observed variable measured in the future

32
Q

What are the four methods for evaluating convergent and discriminant validity?

A

There are four methods to evaluate which measures show convergent and discriminate associations

  • Focused associations
  • Sets of associations
  • Quantifying construct validity (QCV)
  • Multitrait-Multimethod Matrix
33
Q

What is focused associations?

A

Association between test scores and particularly important criteria
A validity generalisation study (VGS) plays a big role as it can evaluate the degree to which an association between a test and an important criterion variable generalizes across the population, etc.
This gives support to convergent validity as it shows the connections.

34
Q

What is a Set of associations

A

Associations between test scores and a wide range of relevant variables
(usually evaluated subjectively via eyeballing the entire set of associations)

35
Q

How do you quantify construct validity?

A
  • By the association between a predicted pattern of convergent and discriminant validity correlations and an actual pattern of convergent and discriminant validity correlations.
    • Has an emphasis on precision and objectivity compared to previous methods.
    • Intends to answer “Does this measure predict an array of other measures in a way predicted by theory?”
36
Q

What is a Multitrait-Multimethod Matrix (MTMMM)

A

Associations between multiple constructs (traits), each of which is measured via multiple methods
Used to interpret all the correlations in a NN
Since NN can be very large, its helpful to use a statistical method

37
Q

What two issues make interpreting validity coefficients challenging?

A
  • Trait variance
    • Shared variance in the test scores due to the same trait being measured can increase the correlation
  • Method variance
    • Shared variance in the test scores due to different traits being measured with the same method can increase the correlation

See image 8.6 for a handy dandy table

38
Q

Based on these two issues, what 4 types of correlation are there?

In the terms of a MTMMM

A
  • Heterotrait-heteromethod correlations
  • Heterotrait-monomethod correlation
  • Monotrait-heteromethod correlations
  • Monotrait-monomethod correlations
39
Q

What are Heterotrait-heteromethod correlations?

A
  • Based on measures of different constructs measured through different methods
  • Sources of variance: Nonshared trait variance and nonshared method variance
  • Weakest expected correlation
40
Q

What are Heterotrait-monomethod correlations?

A
  • Based on measures of different constructs measured through the same method
  • Sources of variance: Nonshared trait variance and shared method variance
  • Moderate expected correlation
41
Q

What are Monotrait-heteromethod correlations?

A
  • Based on measures of the same construct measured through different methods
  • Shared trait variance and nonshared method variance
  • Moderate expected correlations
  • This gives evidence for convergent validity, but must be interpreted in context of the other correlations
42
Q

What are Monotrait-monomethod correlations?

A
  • Based on measures of the same construct measured through the same method
  • Shared trait variance and shared method variance
  • Strongest expected correlation
  • Can be interpreted as reliability, since its basically test-retest (If youve got two result sets)
43
Q

You’ve just made a self-report test for social skills.
Look at image 8.7
What type of correlation does the number in the RED circle represent?

A
  • RED
    • Monotrait-heteromethod correlation.
    • It shows convergent validity.
    • It should be high because of trait variance. (still valid)
44
Q

You’ve just made a self-report test for social skills.
Look at image 8.7
What type of correlation do the numbers in the YELLOW circles represent?

A
  • YELLOW
    • Heterotrait-heteromethod correlations
    • It shows Discriminant validity
    • Correlation should be smaller, since its different traits and different methods.
    • Only correlation should be due to natural relation
    • In a valid MTMMM, it should be smaller than red.
45
Q

You’ve just made a self-report test for social skills.
Look at image 8.7
What type of correlation does the number in the BLUE circle represent?

A
  • BLUE
    • Heterotrait-monomethod correlation
    • It shows discriminant validity
    • Correlation should be lower than red. (In a valid MTMMM)

You may now look at images 8.8/8.9 for examples of more complex

46
Q

What is consequence of test use validity?

A

The degree to which the intended consequences of test use match the actual consequences of test use.

47
Q

What is the first type of evidence for consequences?

A
  • Evidence of intended effects
    • Does the test help in what it was needed for?
    • E.g. does a screening test in personnel selection save time?
48
Q

Whats the second type of evidence for consequences?

A
  • Evidence regarding unintended differential impacts on groups (or adverse impact)
    • Does the use of the test result in unintended benefits for a certain group?
    • Be careful, as the difference may be the result of a genuine psychological difference, rather than as a result of bias

If you want examples, slide 38 has them, but its a bit of yap

49
Q

Whats the third type of evidence for consequences?

A
  • Evidence regarding unintended systematic effects
    • Sometimes the use of a test impacts organizational systems, such as schools and teachers focusing their efforts on what’s tested on the exams, rather than what the student should know in general
    • Another example is MCAT (medicine admission test) shaping how pre-med is taught, however it changed to better reflect being a doctor (more social stuff, ew).
50
Q

What factors affect validity coefficients?

A
  • “True” Associations between constructs
  • Random measurement error and reliability
  • Restricted range
  • Skew and relative proportions
  • Method variance
  • Time
  • Prediction of single events
51
Q

How does the “True” Association between constructs affect the correlation

A

Stronger association between constructs = stronger correlation
Weaker association between constructs = weaker correlation

So if youre doing research on closely related things, obviously you’ll get better validity, vice versa.

52
Q

How does Random measurement error affect the correlation

A

More error = lower reliability = weaker correlation

Remember each test has its own reliability, so uh ohs.

53
Q

How does Restricted range affect the correlation

A

Artificially restricted range = weaker correlation
Since it reduces the potential variability of scores
E.g. GPA being limited to 4.0

54
Q

How does Skew and relative proportions affect the correlation?

A

Differential skew = weaker possible correlation
The skew can be affected by the specific group’s result.
(if the skews don’t overlap, you get worse correlation, even though it might be the group’s fault)

55
Q

How does Method variance affect the correlation?

A

Dissimilar methods = weaker correlation

56
Q

How does Time affect the correlation?

A

Greater time distance (between score sampling) = weaker correlation

57
Q

How does the Prediction of single events affect the correlation?

A

Single events as criterion = weaker correlation

So base your results on multiple samples, not just one (yknow, basic science rules)

58
Q

Other perspectives on validity

A

There are other perspectives on validity, but this was a bunch of yap at the end.
These are the three points, tell me if you want detail about each point cus otherwise id say not worth your time (i hope)
* One perspective says that criterion validity is more important than the interpretation of test scores as a hallmark of test validity
* Another perspective emphasizes the need to learn what test scores mean, rather than the need to evaluate specific hypotheses about test scores.
* The third perspective strongly emphasizes the connection between tests and psychological constructs

59
Q

How is reliability different from validity?

A

You could discuss the reliability of test scores without even being aware of the potential interpretation of test scores or the nature of the trait being measured by the test.
But with validity, its tied to the interpretation of test scores.

PTPR (8 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions