Lecture 1 + WA

Question 1

Why are tests needed to measure things like working memory?

Answer 1

Psychological constructs are unobservable, as such you need test items (a measurement instrument) to quantify the individual differences within the construct and make it observable

Question 2

There are six main ways in which psychological tests differ, which are these? And how?

Answer 2

Content (dependent on construct
Response (open-ended vs close-ended)
Method (individual vs group)
Use (criterion referenced or norm referenced)
Timing (speed tests vs power tests)
The meaning of indicators (formative vs reflective)

Question 3

What is the difference between criterion and norm referenced tests?

Answer 3

Criterion referenced determines if someone passes a certain cut-off and norm referenced copares the scores to the population

Question 4

What is the difference between speed and power tests?

Answer 4

Speed tests have a time limit, the questions are relatively easy and the point is to see how many questions someone can answer within the time limit.
Power tests do not have a time limit and provide a range of difficulty levels to see which questions can be solved

Question 5

What is the difference between formative and reflective measurement?

Answer 5

Reflective measurement assumes the construct to cause differences in the test scores and item responses are indicators of the construct, the items are also necessarily correlated.
Formative measurement assumes the item responses (indicators) define/cause the construct and items are NOT necessarily correlated

Question 6

Is formative or reflective measurement more commonly used in psychological tests?

Answer 6

Reflective

Question 7

What are six challenges in psych measurement and what do they mean?

Answer 7

Complexity
Reactivity (of subjects)
Observer bias (expectation of researcher may affect results)
Composite scores (multiple items need to be combined into one)
Sensitivity (Uknown beforehand, balance between too few and too many categories)
Awareness (test administrators may not know about psychometric qualities of a test)

Question 8

What is the dimensionality of a test?

Answer 8

The number of constructs a test measures

Question 9

What is the difference between a unidemensional and a multidimensional test?

Answer 9

Single construct vs multiple construct measure

Question 10

There are three types of multidimensional tests, which are these and how do they differ?

Answer 10

w/ uncorrelated dimensions (constructs do not correlate, highly unlikely in practice)
w/ correlated dimensions (constructs do correlate, i.e., big-5)
w/ higher-order factor (general construct is correlated to lower constructs, i.e. WAIS-V - general intelligene has smaller subsets)

Question 11

What is factor analysis?

Answer 11

A statistical method to study the dimensionality of a test

Question 12

There are two types of factor analysis, which are these?

Answer 12

Exploratory factor analysis (EFA) = there is no theory about the factor structure
Confirmatory factor analysis (CFA) = There is a clear theory about the factor structure (not covered here)

Question 13

There are four steps for EFA, which are these?

Answer 13

Correlation matrix, checking the eigenvalues, selecting a number of factors and interpreting those factors

Question 14

Why is a correlation matrix needed for EFA?

Answer 14

Correlations between dimensions may indicate commonality between them (i.e., spanish and english being correlated may be due to a higher order language skill being needed)

Question 15

Eigenvalues need to be calculated for EFA, what are these used for?

Answer 15

These are used in determining the amount of factors within EFA

Question 16

How do you determine how many factors there are in EFA?

Answer 16

Through the eigenvalues and a scree plot, the “cut-off” point or inflection point show you where the difference is no longer significant.
i.e., if there is an inflection point at three, there are two factors
see slide 18 lecture 1 for visualization

Question 17

What is the factor loading?

Answer 17

Correlation between item and factor

Question 18

Factor loadings can either be a simple structure or more realistic, what is the difference?

Answer 18

Simple structure means that your dimensions correlate perfectly with their supposed factor, realistic ones show a more diverse correlation across the board (although hopefully still clearly correlate to the correct factor)

Question 19

There are certain tricks one can use when the factor loadings are unclear, which are these?

Answer 19

Orthogonal rotation (varimax or direct oblimin). With varimax you rotate the axis (equally), this will provide clearer factor loadings. with oblimin you reduce the angle, which shows an even clearer factor loading

Question 20

Why are the factor scores useful?

Answer 20

They reduce data (can be used in t-test for instance), they have advantages over the sum scores and they avoid multicollinearity (independent variables correlate)

Question 21

Within oblique rotation there are two types of matrix’, which ones and how do they differ?

Answer 21

Pattern matrixes control the factor loadings for correlation, this makes it much easier to intepret
Structure matrixes do NOT control for this correlation (and thus shows shared variance), making them more complex to interpret

Question 22

What is a contra-indicative item?

Answer 22

Higher scores on the item indicate lower levels of the construct

Question 23

What is a factor correlation matrix? What does it ideally look like?

Answer 23

Shows the correlation between factors, ideally you do NOT want the factors to be correlated with each other

Question 24

The human development index (HDI) is a composite measure of human development consisting of health, education and income. It is used by the United Nations to measure the social and economic development per country.

Are the scores on a HDI test reflective/effect indicators or formative/causal indicators, and why?

Answer 24

Formative/causal indicators, because the scores on health, education and income indicators make up a country’s HDI level

Question 25

Kelly has to take an arithmetic test. In order to pass, she needs to answer 70% of the questions correctly. The test is an example of an:

Answer 25

Criterion-referenced test

Question 26

A researcher conducts a longitudinal study measuring the language ability of a group of children. Over the past years, during various measuring occasions, the language ability of the girls tended to increase more than that of the boys. Therefore, the researcher expects that also at the current measuring occasion, the girls’ language ability has increased more than the boys’.
What challenge of psychological testing might come into play here?

Answer 26

Observer/scorer bias

Question 27

Tom participates in an experiment. During this experiment, he has to fill in a questionnaire about sustainable behavior. Tom doesn’t want others to think of him as an environmentally unfriendly person. How do we call it when Tom’s awareness that he is being assessed influences his answers on the questionnaire?

Answer 27

Participant reactivity

Question 28

What are three qualities of individual differences?

Answer 28

1. Individual differences are psychological differences between people. In order to measure them, we need samples of people that differ regarding the behavior that is measured
2. By measuring individual differences among people, we identify variations in behavior across people. This helps us to investigate the full range of human behavior
3. Individual differences are important in all fields of psychology

Question 29

Between intelligence, age and personality, on which do you expect the most individual differences in a sample of first-year university students of a degree?

Answer 29

Personality

Question 30

Nathan conducts research into social anxiety for his master thesis. He uses a social anxiety questionnaire to assess the social anxiety levels of a student sample. The results show that participant 57 has a z score of 0.7. What does that mean?

Answer 30

That participant 57 has a social anxiety test score that is 0.7 standard deviations above the mean

Question 31

Scores on the Wechsler Adult Intelligence Scale IV (WAIS-IV) have a mean of 100 and a standard deviation of 15. What kind of scores are these?

Answer 31

Standardized scores

Question 32

Standardized scores, Normalized scores, Z-scores and standard scores, what is the difference?

Answer 32

Z scores and standard scores are the same (i think), they indicate whether a raw score falls above or below the mean and the distance this is

Standardized scores are a score that have been transformed to have a specific mean and standard deviation

Normalized scores are test scores that have been transformed to reflect their relative standing within a normally distributed set of scores

Question 33

What is an necessary assumption when linking z-scores to percentiles?

Answer 33

Normally distributed individual differences

Question 34

With a z-score of 1.17 and an area of .3790, what would be the percentile rank?

Answer 34

87.90 (just add .5 if positive or substract the area from .5 for a negative)

Question 35

In what situation do we use normalization transformation to obtain normalized scores?

Answer 35

When the raw test scores are NOT normally distributed

Question 36

If a distribution is right skewed, is it larger or smaller than 0?

Answer 36

Right skew = >0

Question 37

What are three components of the correlation?

Answer 37

It is a measure of linear association, it gives clear info about the direction of the association, it is a number btwn -1 and 1 and it gives clear info about the magnitude of the association

Question 38

What are two components of the covariance?

Answer 38

Clear info about the direction of the association and it is a measure of linear association

Question 39

What is the relation between correlation and covariance?

Answer 39

A correlation is a standardized covariance

Question 40

If a person has a z score of 1.13 and an area of .3708 what can you conclude?

Answer 40

87.08% of the other persons in the sample scored lower than that person

Question 41

For which of the test(s) is it meaningful to compute a total test score (i.e., aggregating over all dimensions)?
Note: from the multidimensional ones

Answer 41

one with a higher order factor

Question 42

The Novaco Anger Scale is part of the two-piece Novaco Anger Scale-Provocation Inventory (NAS-PI; Novaco, 1994) and consists of 48 items measuring an individual’s anger experiences. The NAS has three subscales: cognitive, arousal and behavior. The test produces scores on each of the subscales, as well as a total test score.

What kind of test is the Nocavo Anger Scale?

Answer 42

Multidimensional test with a higher-order factor

Question 43

Indicate how many test scores each of the below tests produces (give a round number)
- A unidimensional test with 20 items
- A multidimensional test with 4 uncorrelated dimensions and 40 items
- A multidimensional test with 4 correlated dimensions and 40 items
- A multidimensional test with 4 dimensions and one higher-order factor and 50 items

Answer 43

1, 4, 4 and 5, respectively

Question 44

What is a critique of using the eigenvalues-greater-than-one rule to determine the number of factors in factor analysis?

Answer 44

It likely results in an overestimation of the number of factors

Question 45

Look at figure 1.1, what can you conclude about the dimensionality of the data that these analysis are based on?

Answer 45

The four factors are correlated, which means that the data are correlated multidimensional

Question 46

What are two differences between factor, pattern and structure matrixes?

Answer 46

1. The loadings of the factor matrix are obtained before rotation. The loadings of the pattern and structure matrix are obtained after rotation.
2. The pattern matrix controls for interfactor correlations when estimating the association between items and factors. The structure matrix does not.
3. The factor matrix is not rotated and the factors are uncorrelated, the pattern matrix controls for correlations and the structure matrix does not control (so they are correlated)

Question 47

A difference between oblique rotation and orthogonal rotation?

Answer 47

Oblique rotation generally produces factor loadings that are closer to simple structure than orthogonal rotation

Question 48

Which matrixes belong to which rotation?

Answer 48

Orthogonal = factor matrix (?), oblique = structure and pattern