Validity

Question 1

Define Validity.

Answer 1

The extent to which the test measures what it purports to measure

Question 2

What is Construct Validity?

Answer 2

The meaning of a test score made possible by knowledge of the pattern of a relationships it has with other variables and the theoretical interpretation of those relationships

Question 3

Explain Face Validity.

Answer 3

The extent to which a measurement or assessment appears, on the surface, to measure what it intends to measure.

It is a subjective judgment based on whether a test or measurement looks like it is measuring the intended construct or concept.

Question 4

What is Content Validity?

Answer 4

The extent to which a measurement or assessment covers all the important aspects or content of the construct being measured.

It is concerned with whether the items or questions included in the measurement tool represent the full range of the construct being studied.

Question 5

What are the differences between Face Validity and Content Validity?

Answer 5

1. Face validity is a subjective judgment based on appearance or perception, while content validity is an objective evaluation based on the coverage of important content
2. Face validity is a surface-level evaluation of whether a measurement “looks like” it measures the intended construct, while content validity involves a thorough analysis of whether the measurement adequately represents the full range of the construct
3. Face validity is an initial and quick assessment, often used as a rough indicator, while content validity involves a more rigorous and systematic assessment to ensure comprehensive coverage.

Question 6

What is Predictive Validity?

Answer 6

The extent to which a score on a psychological test allows a statement about standing on a variable indexing important social behaviour independent of the test.

Question 7

What is Concurrent Validity?

Answer 7

A form of predictive validity in which the index of social behaviour is obtained close in time to the score on the psychological test

Question 8

What is a method used to determine one’s predictive validity?

Answer 8

Regression

Question 9

How is the correlation between test and criterion often evaluated?

Answer 9

r^2

Question 10

Under Coefficient of Determination, what does correlation of 0.3 suggests?

Answer 10

This suggests that the scores on the test account for 9% (r^3) of the variance in the criterion

Question 11

Explain Incremental Validity.

Answer 11

An extension of predictive validity

It pertains to the evaluation of unique contribution a test has over other existing tests in predicting an outcome

Question 12

Explain Standard Error of Estimate.

Answer 12

An index of the amount of error in predicting one variable from another

Question 13

What are the 4 types of decision-theoretic approach?

Answer 13

1. Valid Positive Decisions
2. Valid Negative Decisions
3. False Positive Decisions
4. False Negative Decisions

Question 14

Explain valid negative decision (C)

Answer 14

Assumption: Does not have characteristic
Reality: Does not have characteristic

Question 15

Explain valid positive decision (B).

Answer 15

Assumption: Have characteristic
Reality: Have characteristic

Question 16

Explain false negative decision (A).

Answer 16

Assumption: Does not have characteristic
Reality: Have characteristic

Question 17

Explain false positive decision (D).

Answer 17

Assumption: Have characteristic
Reality: Does not have characteristic

Question 18

How to calculate the Base rate (BR) of criterion behavior in the population of interest?

Answer 18

(A + B) / N

Question 19

What is Base rate?

Answer 19

The proportion of individuals who have the characteristic in a population (or sample of)

Question 20

What is Selection ratio?

Answer 20

Proportion of individuals selected (or categorised positively) based on test cut-off score

Question 21

How to calculate selection ratio?

Answer 21

(B + D) / n

Question 22

How to calculate valid positive rate?

Answer 22

B / n

The number of valid positives divided by the total number of persons tested

Question 23

How to calculate hit rate?

Answer 23

(B + C) / n

The sum of number of valid positives and valid negatives divided by the total number of person tested

Question 24

Define test sensitivity.

Answer 24

B / (B + A)

How well does the test do in identifying those who have the characteristic

Question 25

Define test specificity.

Answer 25

How good is the test in ruling out those who do not have the characteristic

C / (C + D)

Question 26

What is Multitrait-multimethod Matrix?

Answer 26

A correlation matrix of scores from various tests administered via different methods (i.e., self-report, observation)

Question 27

If there is a low correlation observed on two tests that are supposed to be measuring the same construct, what does it imply?

Answer 27

It suggests that the test has failed convergent validity because tests measuring the same construct should have a high correlation.

Question 28

What is criterion validity?

Answer 28

It focuses on the overall construct that the measure’s scores are supposed to be indicative of, rather than the actual content of the measure.

(gold-standard)

Question 29

Explain what Y = mX + C each represents.

Answer 29

X = linear relationship between test/predictor
Y = Criterion

This allows for the prediction of Y through sheer knowledge of the X value itself

Question 30

Explain Factor Analysis.

Answer 30

It is a mathematical method of summarising a matrix of values (such as the inter-correlation of test scores) in terms of a smaller number of values (factors) from which the original matrix can be reproduced

Question 31

What is the purpose of Rotation?

Answer 31

It helps to maximize the loadings on one factor and minimizes loadings on another

Question 32

List down 2 different types of Rotation techniques.

Answer 32

1. Orthogonal Rotations
2. Oblique Rotations

Question 33

What is the angle for Orthogonal rotations and are they correlated?

Answer 33

90 degrees; not correlated

Question 34

What is the angle for Oblique rotations and are they correlated?

Answer 34

< 90 degrees; correlated to a certain extent

Question 35

What does h^2 refers to?

Answer 35

It refers to the proportion of variance (multiply by 100 to obtain % of variance)

Question 36

When h^2 = 0.9832 for a specific item, what does it mean?

Answer 36

98% of the specific item’s score variance is accounted by Factor A and B combined

Question 37

How do we calculate V for a Factor?

Answer 37

Square each of its loadings and summing them up, then dividing by the number of items

Question 38

V = 0.54 means?

Answer 38

The factor accounts for 54% of the entire test’s score variance

Question 39

What is Exploratory Factor Analysis?

Answer 39

This method of factor analysis is typically used during early test development

It lets the data “speak for itself”, where software decides on the dimensional cuts that best accounts for most of the variance in the data

Question 40

What is Confirmatory Factor Analysis?

Answer 40

Used when you have an existing measure where you know what it is supposed to measure

You tell the software that you are supposed to see a specific factor structure just indicating one construct, and the software will tell you if it fits well with the data obtained

Question 41

What are the 2 types of EFA?

Answer 41

1. Principal Components Analysis (PCA)
2. Principal Axes Factoring (PAF)