Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

PTPR > Final WA > Flashcards

Final WA Flashcards

1
Q

Of what is validity a property of?

A

A psychological test

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

When do we speak of construct underrepresentation?

A

When a test does not include the entire range of content of the construct that it intends to measure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Rachel analyzes an anxiety test and finds that the items form two factors. To what type of validity evidence does this finding contribute?

A

Internal structure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Alex developed a questionnaire for her research about neuroticism. She wants to know if her questionnaire is reasonably valid, that is, if it has sufficient construct validity. To this end, Alex asks researchers at her university – who are experts in this field – to rate each of the items in terms of how essential they are to the test.

What type of validity evidence is Alex evaluating here?

A

Test content

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Is criterion validity a seperate category from convergent validity?

A

No, they are both a part of association with other variables

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Can criterion validity be split up into predictive and discriminant validity?

A

No, predictive and concurrent

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

According to theory, neuroticism is positively correlated with anxiety. Keeping this association in mind, what evidence do you need to find that will indicate good associative validity of a neuroticism questionnaire?

A

Evidence on convergent validity

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Consequential validity is about…

A

The uses of tests scores

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

According to wikipedia, the definition of arithmetic is:
“Arithmetic is an elementary part of mathematics that consists of the study of the properties of the traditional operations on numbers—addition, subtraction, multiplication, division, exponentiation, and extraction of roots.”

John creates an arithmetic test with items about addition, subtraction, multiplication, division, and exponentiation. What does this test suffer from?

A

The test suffers from construct underrepresentation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

In an exam on the history of psychology, students can pass the exam simply by choosing the most elaborate answer option for each item. Which type of validity is being threatened?

A

Response process validity

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

In an exam on developmental psychology, students that have children of their own have a higher passing rate on the exam (85% of these students passes the exam) compared to students without children of their own (65% of these students passes the exam)

Which of the below causes for this difference forms a threat to consequential validity?

  1. This difference is due to the students with children being able to relate the course material to their own children making the exam easier for them
  2. This difference is due to the students with children studying harder for the exam than the students without children
  3. This difference arose as the students with children took the exam in the morning as the first exam of the day, while the students without children took the exam in the evening after two other exams
A

(1 and 2 are both fair. Studying hard and understanding something better because you can relate it to something in real life are both intended consequences, 3 is not.)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

In the psychological methods bachelor’s program, there are courses on programming, advanced statistics, and philosophy of science. After this program, students can enroll into the Behavioral Data Science (BDS) master track.

Since 2017, students are only allowed into the track if they showed
-that they attended programming courses
-that they attended advanced statistics courses

Ever since this selection procedure was invoked, less students attended the philosophy of sciences courses from the psychological methods program.

What kind of consequential validity evidence is this?

A

Evidence of unintended systematic effects

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

A researcher wants to study the structural validity of a test measuring social anxiety. The test measures cognitive and affective aspects of social anxiety. The scree plot shows an inflection point at 2. On the basis of that plot, what would you conclude about the validity of this measurement.

If a validity is being threathened, which type?

A

These results show evidence against the structural validity of this measurement instrument

(bc only one factor is found, there should theoretically be two)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

A researcher wants to study the structural validity of a test measuring dyslexia. The test measures reading and writing aspects of dyslexia. In a factor analysis he finds that the factors correlate .901 with each other. On the basis of these results, what would you conclude on the validity of this measurement instrument?

If a validity is being threathened, which type?

A

These results show evidence against the structural validity of this measurement instrument

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

For a sample of secondary school students, a researcher correlates the scores on a career-choice test to the actual university program that the students choose a year later. What kind of validity is being studied?

A

Predictive validity

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

A research establishes the correlation between a test on openness to experience and the variable “number followers on Instagram”.
What kind of validity does the researcher try to establish?

A

Concurrent validity

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Why is it so important to use a reliable criterion measure when evaluating a validity correlation?

A

Because the validity coefficient will be attenuated when the criterion measure is not reliable

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What is the relationship between reliability, validity and measurement error?

A

High measurement error –> poor reliability –> attenuated validity coefficient

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

We want to calculate the convergent validity correlation between two tests: A and B. On test A, a lot of people obtained the highest score, while on test B the scores are more normally distributed

How does this difference affect the validity coefficient?

A

Test A doesn’t reflect the full variability of behavior, as the range is restricted, which weakens the validity coefficient

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

We are interested in the validity correlation between two tests, test A (scores between 1 and 10) and test B (scores between 1 and 10) . However, before calculating the correlation between tests A and B, we select all subjects that score 3 or higher on test A. We then calculate the validity correlation between test A (scores 3 and up) and test B.

What will happen to the validity coefficient?

A

The validity coefficient will decrease, because the range of test A is restricted

21
Q

Liz wants to investigate the validity of the attachment questionnaire she made for her research. To do that, she looks at the convergent validity correlation between her test and a self-esteem test, since theory suggests that secure attachment is correlated with high self-esteem. The self-esteem test is skewed to the right, what can you conclude about the validity coefficient?

A

The validity coefficient will be reduced, because the distribution of the self-esteem test is skewed

22
Q

Liz wants to investigate the validity of the attachment questionnaire she made for her research. To do that, she looks at the convergent validity correlation between her test and a self-esteem test, since theory suggests that secure attachment is correlated with high self-esteem. Below are the probability distributions of the data she collected (i.e., respondents who filled in both questionnaires).

Liz discovers that in her sample, there are much more people with high self-esteem than with low self-esteem. How will this affect the validity?

A

As the self-esteem group sizes are not equal, the criterion variable is skewed, which leads to a lower validity coefficient

23
Q

What is a risk of using predictive validity correlations when evaluating validity?

A

Underestimation of validity coefficients, because the variables are measured at different points in time

24
Q

Gerry wants to evaluate the validity of his questionnaire about worrying. It was filled in by 150 respondents, who answered questions about how much they usually worry in various situations. Gerry discovered that his measure is affected by quite some measurement error. As a criterion variable, Gerry uses nervousness. From theory he knows that the correlation between worrying and nervousness is high. For his analysis, nervousness was measured through observation: the nervous behavior of the respondents was observed while they were filling in the questionnaire (i.e., and expert rated to what degree the respondents were nervous during the test session).

Which two factors weaken the validity coefficient in the case above?

A

Low reliability and prediction of single events

(bc using a single event as a criterion will weaken the correlation)

25
Q

Which six factors, not including measurement error, affect the validity coefficient and why/how?

A

Association btwn constructs (stronger association = ^ correlation)

Restricted range (weakens correlation because it limits true variability) im pretty sure this also relates to weak discrimination if the range is restricted

Skew (if there is a difference in skew it weakens correlation bc without the distributions being similar the correlation is capped and can never be 1 (bc 1 can only be if something correlates with itself/aka same value).
> note that skew also happens when groups are unequal (or can happen)

Method variance (dissimilar methods = weaker correlation bc correlations based on a single method are inflated due to bias)

Time (greater time distance = weaker correlation bc of variables during interval)

Prediction of single events (single events = weaker correlation due to unreliability of measuring a single event/response)

26
Q

Using factor analysis, what aspect of validity is the researcher studying?

A

Internal Structure

27
Q

consider a MTMM matrix, if we focus on the substantial correlations between OS_self on the one side and WS_self and EM_self on the other side (respectively 0.42 and 0.40). (r btwn os_self and the other two)

What does this large correlation indicate?
Look at figure 1.2

A

These correlations indicate that the test has poor discriminant validity

28
Q

We focus on the substantial correlations between OS_self on the one side and WS_self and EM_self on the other side (respectively 0.42 and 0.40).

How are these correlations called?
figure 1.2 might help, but not needed

A

Heterotrait-monomethod correlations

29
Q

In an exam on developmental psychology, students that have children of their own have a higher passing rate on the exam (85% of these students passes the exam) compared to students without children of their own (65% of these students passes the exam)

Which of the below causes for this difference indicates DIF?

  1. This difference is due to the students with children being able to relate the course material to their own children making the exam easier for them
  2. This difference is due to the students with children studying harder for the exam than the students without children
  3. This difference arose as the students with children took the exam in the morning as the first exam of the day, while the students without children took the exam in the evening after two other exams
A

3

the other two are fair game

30
Q

Which validity is threathened when DIF is present?

A

consequential validity

31
Q

Two arithmetic items, item A and B are compared on their difficulty: Item A has a lower item difficulty parameter value as compared to item B. What can you conclude?

A

Item B will have a lower proportion of correct responses than item A

32
Q

In an IRT analysis the item difficulty of item 9 is found to be 0.78. What can you conclude?

A

People with a trait level of 0.78 have a 50% chance of answering item 9 correctly

33
Q

Karin and Sophia are administered a spelling test. Sophia has a spelling ability of θ = 1.5 and Karin has a spelling ability of θ = 0.6. Their teacher calculates the probability of a correct response on item 5 for Karin and for Sophia. The teacher uses a 2PL model. The probability that Sophia answers item 5 correctly is 0.8 and the probability that Karin answers item 5 correctly is 0.7.

How can it be that these probabilities are so similar, when in fact Karin’s and Sophia’s ability levels are quite different?

A

Item 5 is not good at differentiating between people with various ability levels

34
Q

Say you developed a test to measure risky behavior in adolescents. The test consists of 16 items, that are scored on a 7-point Likert scale. The items are not equally good at differentiating between the risky behavior of the test-takers.

What model would you use to estimate the item and person parameters?

A

GRM

35
Q

If you have 16 items that are scored on a 5-point Likert scale. How many difficulty parameters does a GRM model include for each item? Give a round number

A

4

36
Q

In a 2PL with 15 items. How many parameters are there in total? Give a round number

A

30

37
Q

In a 1PL with 20 items. How many parameters are there in total? Give a round number

A

20

38
Q

In a 3PL with 25 items. How many parameters are there in total? Give a round number

A

75

39
Q

In a GRM with 10 items that use a 5-point Likert scale. How many parameters are there in total? Give a round number

A

50

40
Q

Look at figure 1.3, from which model do these curves originate (only 1)?

A

2PL

the curves differ in their slope

41
Q

Look at figure 1.3, what is the approximate value for the difficulty of item 1?

A

-1

it’s like the middelish of the slope

42
Q

Look at figure 1.4, which model is this?

A

3PL

guessing value present

43
Q

Look at figure 1.4, which item has the worst discrimination and which one has the best?

A

worst = 1 and best = 3

44
Q

What is the problem with a test that only has easy items, with regard to test information?

A

The test provides little information at high trait levels, since the items do not discriminate well among people with high trait levels

45
Q

In a 2PL, what can you conclude about the item information if an item has a difficulty of 0.65?

A

At a trait level of 0.65, the item provides the most information

46
Q

Consider the following situation, for a spatial ability item, older people score lower than younger people.

Which of the below indicates that this item has DIF?

  1. The older people score lower because the item used a picture from Instagram that most of the older people are unfamiliar with, while most of the younger people have seen that picture before.
    check
  2. The older people score lower because the test was too difficult for most of them while for most of the younger people the test was appriopriate.

3.The older people score lower because they have a lower spatial ability level than the younger people

A

1.

2 indicates that the latent trait was just lower for older people and 3 indicates that there is just a mean difference on the latent trait

47
Q

What is the main benefit of using a CAT instead of a conventionnal test?

A

A CAT requires administration of fewer items

48
Q

How would you select the items for a criterion references test? (difficulty wise)

A

You select mostly items of with difficulty close to the cut-off point

PTPR (4 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2025 Bold Learning Solutions. Terms and Conditions