Lecture 1 - ch1 Introduction to psychometrics

Question 1

Why is knowledge about psychological testing valuable?

Answer 1

• Without solid understanding of basic principles of psychological measurement = risk of misinterpreting or misusing the information derived from psychological tests
• If something is not measured or is not measured well, then it cannot be studied with any scientific validity
• Know when to accept of believe test scores, when to question the use and interpretation of test scores

Question 2

What are psychological tests used for?

Answer 2

• Psychologists, educators - as an instrument to measure observable events in the physical world
• Observe human behaviour as a way of assessing unobservable, psychological attributes
• E.g. difference in behaviour → difference in working memory

Question 3

What are hypothetical contructs/latent variables?

Answer 3

Theoretical psychological characteristics, attributes, processes, states that can’t be directly observed

Question 4

What is a psychological construct?

Answer 4

Theoretical concept of psychological differences between individuals
→ Unobservable
E.g. working memory, extraversion…

Question 5

What are operational definitions?

Answer 5

The operations or procedures that we can use to measure the hypothetical constructs
Operationalisations = turning the hypothetical contruct into measurable observation

Question 6

What is a psychological test?

Answer 6

Measurement instrument to quantify individual differences in the construct (key interest of psychologists)
- It is a way of operationalisation of the psychological contruct

Question 7

How do we link unobservable and observable variables?

Answer 7

Make an inference, observable behaviour (number of recalled numbers) is systematically related to unobservable psychological attribute (working memory)
- If this inference is reasonable → our interpretation has a degree of validity
↪ Actually measuring what we want to measure

Question 8

What are the three components of a psychological test according to Cronbach’s definition?

Answer 8

'’A systematic procedure for comparing the behavior of two or more people”
1. Tests involve behavioural samples of some kind
2. The behavioural samples must be collected in some systematic way
3. The purpose of the test is to detect differences between people - could be modified to include a comparison of performance by the same individuals at different times or situations

Question 9

What are two important features of Cronbach’s definition of psych tests?

Answer 9

1. Generality
2. General purpose

Question 10

What is meant by generality of Cronbach’s definition?

Answer 10

Generality of:
1. what a psych. test might be → laboratory situations, paper tests…
2. the type of information a test produces → numbers representing the amount of some psych. attribute possessed by a person, categorical data

Question 11

What is the general purpose of psychological tests based on Cronbach’s definition?

Answer 11

Tests must be capable of comparing the behaviour od different people (interindividual difference) or the behaviour of the same individuals at different points in time or under different circumstances (intraindividual differences)

These differences on test performance contribute to test score variability → a necessary component of measurements of psych attributes

Question 12

What are the dimensions by which we judge the type of a test?

Answer 12

1. Content
2. Response required
3. Method of administration
4. Use
5. Timing
6. The meaning of indicators

Question 13

What does the content of a test depend on? Examples?

Answer 13

Depends on the contruct to be measured
- e.g. aptitude, achievement, intelligence, personality…

Question 14

The different types of response required?

Answer 14

1. Open-ended
2. Closed-ended

Question 15

Method of administration

Answer 15

1. Individual → one person at a time
2. Group → multiple people at the same time

Question 16

What are the two uses (purposes) of a test?

Answer 16

1. Criterion-referenced → determines if someone passes a certain cut-off (e.g exam- sufficient knowledge showed?)
   ↪ cutoff score established as a criterion which sorts people into those who exceed it and those who don’t
2. Norm-referenced → compare a person’s score to population (reference sample/’‘average person’’)

Distinction often blurred in practice when norm-referenced scores are used as cutoff scores

Question 17

How do we divide the type of a test based on its timing?

Answer 17

1. Speed test (time limit) → relatively easy questions, see how far one comes
2. Power tests (no time limit) → different difficulty level, number of correct answers counted (expected to complete the whole test)

Question 18

The meaning of indicators

What are reflective (effect) indicators?

Answer 18

Construct is assumed to cause differences in the test scores
↪ Item responses are indicators of the construct (unobservable construct ► observable items )
- Items are necessarily correlated → higher values on the construct increases scores on all items
- Most commonly used

Question 19

The meaning of indicators

What are formal (causal) indicators?

Answer 19

The item responses (indicators) define the contruct
- Items are not necessarily correlated
↪ Items independently contribute to the construct

E.g. Socioeconomic status (contruct) is determined by person’s income, education level, occupation (items)

Question 20

What is a test battery?

Answer 20

Bundeled tests administered together but aren’t necessatily designed to measure a single psychological attribute

Question 21

What is psychometrics?

Answer 21

Science concerned with evaluating the attributes of psychological tests
↪ Those attributes are:
1. The type of info generated by the use of psych tests
2. The reliability of data from psych tests
3. Issues concerning the validity of data obtained

Question 22

Brief history

What are two key foundations psychometrics is build on?

Answer 22

1. The practice of psych testing and measurement - in China 2000-4000 years back
   ↪ increase in 19th century since researchers began systematically measuring qualities and responses of individuals in studies
   ↪ more in 20th - intelligence and personality tests
2. Related historical foundation - developemnt of particular statistical concepts and procedures
   ↪ Pearson, Galton (founding father of psychometrics), Spearman

Question 23

Brief history

When did psychometrics become its own field? How was it developing since?

Answer 23

• Around 1930s-40s - journal Psychometrika published regularly + Central test theory (CTT) articulated
• 50s - crucial concept of test validity was undergoing development with reconstruction in 90s
• 70s - CTT expanded into generalizability theory + alternative developed: Item Response Theory

Question 24

What are the effects of challenges to measurement in psychology?

Answer 24

Reduce measurement accuracy - affect our confidence in our understand and interpretation of behavioural observations

Question 25

What are the main challenges to psychological measurement?

Answer 25

1. Complexity
2. Reactivity
3. Observer bias
4. Composite scores
5. Sensitivity
6. Awarness

Question 26

Complexity

Answer 26

Psychological phenomena are complex, high-dimensional concepts
- Difficult to identify and capture the important aspects in a single number or score (‘‘You can’t reduce people to a number’’)
- You need to carefully isolate a psychological dimension (construct) and measure it

Question 27

Participant reactivity

Answer 27

People respond differently when they know they’re being measured
- They react in ways that obscure the true psychological characteristic
- Demand characteristics and Social desirability

Question 28

What are demand characteristics?

Answer 28

Participants try to figure out the purpose of the study, changing their behaviour to accomodate the researcher

Question 29

Social desirability

Answer 29

Change their behaviour to try to impress the person doing the measurement or to convey poor measurement (malingering)

Question 30

Observer Bias

Answer 30

Researchers sometimes bring their own biases and expectations to their study which might affect the test

Question 31

Composite scores

Answer 31

Multiple item scores need to be combined into one score
- Should we maybe weigh the items differently?

Question 32

Score sensitivity

Answer 32

Sensitivity = measure’s ability to discriminate between meaningful amounts of the dimension being measured
- Unkown beforehand how sensitive a scale should be
- Too few response categories: may miss out on individual differences
- Too many response categories: can’t meaningfully distinguish the different categories

Question 33

Awareness

Answer 33

Many test administrators don’t know abou the psychometric quality of the test
Knowledge of psychometric properties should inform the development or selection of a test

Question 34

What is the purpose of measurement in psychology? Why is it important?

Answer 34

Identify and quantify the psychological differences that exist between people, over time or across conditions
- The person’s score can be understood only in the context of the test’s ability to detect differences
- These differences must be relevant to the underlying psychological attribute = validation process

Question 35

What is differential psychology? What does it contrast to?

Answer 35

The study of individual differences
Psychometrics does depend on those difference (Galton’s primary interest) but it doesn’t mean that psychometrics is not important for other fields of psychology

For example, differential psychology is contrasted to experimental psychology (focus on the average person rather than differences)
↪ need the ability to identify and quantify variability to connect it to psychological phenomena

Question 36

What is the general definition of psychometrics?

Answer 36

Psychometrics is the study of the operations and procedures used to measure variability in behavior and to connect those measurements to psychological phenomena.