Validity

Question 1

What is the difference between reliability and validity?

Answer 1

Reliability tells us about the consistency of the measurement while validity tells us about the accuracy.

Question 2

What type of validity aims to show that the test appears to be testing what it set out to test?

Answer 2

Face validity.

Question 3

What is predictive and concurrent validity aiming to measure?

Answer 3

How well a test estimates scores on a criterion.

Question 4

What is the difference between predictive and concurrent validity?

Answer 4

Predictive aims to predict how a person will perform on a future criterion, while concurrent aims to predict how they will go on a test performed at the same time.

Question 5

What is content validity aiming to measure?

Answer 5

How well a test covers a representative sample of the domain being measured.

Question 6

What is meant by construct and how do we observe them?

Answer 6

A construct is a hypothetical concept that allows us to make more sense of behaviour. We use operations to observe constructs and tests can be considered operations.

Question 7

For validity, do we care more about how large the coefficient is or if it is significant?

Answer 7

Significance matters more than the size of the coefficient.

Question 8

What size coefficient do we want for validity tests?

Answer 8

.2-.5

.2 is the absolute minimum.

Question 9

What is the standard error of estimate telling us

Answer 9

The amount of error between our predicted scores and our actual scores. Think of a regression line.

Question 10

How do we estimate the 95% CI around a predicted score?

Answer 10

We use the standard error of estimate.
Y+/-1.96SEest
This is different to an obtained score!!!

Question 11

What is the difference between measuring a CI around a predicted score and an observed score?

Answer 11

Predicted scores use SEest (because the error is predicted vs obtained)
Obtained scores use SEM (as we want to find out what the true score was)

Question 12

What is the difference between SEest and SEM?

Answer 12

SEest looks at the error between predicted score and obtained score, while SEM looks at error between obtained score and true score.

Question 13

If a prediction criterion is 80 and a person’s score 95% CI ranges from 73-82, would we accept that candidate?

Answer 13

Yes, because their true score could be 80 according to the CI.

Question 14

What is a cutting point?

Answer 14

A cutting point is a score that will separate two groups. People who are above will be accepted, and those who fall below will be rejected.

Question 15

What are the factors that affect the validity coefficient?

Answer 15

Low sample size
Restrictions of range (selectivity problem)
Non-linear relationship between test scores and criterion
Problems with criterion

Question 16

How does the restriction of range affect validity?

Answer 16

restriction of range means we aren’t getting the full range of scores. In essence, we aren’t getting a normative sample. May be testing a group who has already passed the criterion, so upper range restriction. Or self-selection means people won’t complete the test, so skewed with conscientious people.

Question 17

What is the weakest part of criterion-related validity?

Answer 17

The criterion!
We can get criterion contamination where the assessor knows the person will do well so they are biased.
OR
Criterion requirements change, so measuring something different to before.

Question 18

Which two types of validity aren’t empirically established?

Answer 18

Content and face validity.

Question 19

How we make sure we are achieving high construct validity?

Answer 19

Make sure the test is driven by current research and theories.

Question 20

How do we test construct validity?

Answer 20

Correlate scores with other constructs or behaviours or groups. Looking for convergent or divergent results.
Age differentiation
Criterion groups (high and low scoring groups)
Relationship to behaviour