L1: Intro to psych measurement Flashcards
1
Q
What is a psychological construct? + give an example
A
aka latent variable
- theoretical concept of psych differences between individuals
- unobservable
ex: working memory
2
Q
What is a psychological test?
A
- Measurement instrument to quantify individual differences in the psych construct
- Observable
- operational definitions/ operationalizations
3
Q
what are the 3 important components of a psychological test?
A
- involve behavioural samples of some kind
- samples are collected in a systematic way
- purpose is to detect differences between ppl (or within a person across time or situations)
4
Q
What’s the difference between a psychological construct and a psychological test?
A
psych construct = unobservable
psych test = observable
5
Q
How do psych tests differ?
A
- content (depends on the construct being measured)
- response required (open ended vs closed ended)
- method of administration (individual vs group)
- use (criterion referenced vs norm referenced)
- timing (speeded vs power)
- the meaning of indicators (formative vs reflective)
6
Q
What are criterion referenced vs norm referenced psych tests used for?
A
criterion referenced: to determine if someone passes a certain cut-off (ex: exam)
norm referenced: to compare score to population (aka the reference sample) (ex: intelligence test)
7
Q
What are the different types of timing that can be used for psych tests?
A
speed test (time limit): relatively easy questions, see how far one comes
power tests (no time limit): different difficulty level, see what one can solve
8
Q
What is formative measurement?
A
- the item responses (indicators) define the construct
- items are not necessarily correlated (items independently contribute to the construct)
- ex: the value of a car (construct) is determined by its age, condition, size (items)
9
Q
What is reflective measurement?
A
most psych tests involve reflective measurement:
- construct is assumed to cause differences in the test scores
- item responses are indicators of the construct
- items are necessarily correlated (higher values on the construct increase score son all items)
- ex: differences in arithamtic skills cause differences in how well children solve 18+6
10
Q
What are some challenges in psych measurement?
A
- Complexity of psych concepts
- Participant reactivity
- Observer expectancy/bias
- Composite scores (++ items scores need to be combined into one score)
- Score Sensitivity: (unknown beforehand how sensitive a scale should be)
- (lack of) Awareness of psychometrics (by for ex test administrators)
11
Q
What do you need to watch out for in the score sensitivity of a test?
A
- Too few response categories: you may miss out on individual differences
- Too many response categories: people can’t meaningfully distinguish the different categories
not known beforehand how many there should be
12
Q
What is participant reactivity?
A
ppl respond differently as they know that they are being measured
-> threatens the validity of the measure
13
Q
What is observer expectancy bias?
A
expectation of the researcher may affect the test
14
Q
define (score) sensitivity
A
a measures ability to discriminate between meaningful amounts of dimension being measured
15
Q
what are the different forms of participant reactivity?
A
- demand characteristics
- social desirability
- malingering
16
Q
What is malingering?
A
when ppl change their behaviour to convery a poor impression to the person doing the measurment
17
Q
What are demand characteristics?
A
participants change their behaviour to accommodate the researcher
18
Q
What is the social desirability effect?
A
change their behaviour to try to impress the person doing the measurement
19
Q
define psychometrics
A
science concerned w evaluating the attributes of psych tests
20
Q
what are the main test attributes that psychometrics care about?
A
- type of info generated by the use of psych tests
- reliability of data obtained from psych tests
- validity of data obtained from psych tests
21
Q
define variability
A
degree of differences within a set of test scores or among the values of a psych attribute
22
Q
define covariability
A
degree to which varaibility in one set of scores corresponds with (or is consistent with) variability in another set of scores
23
Q
define interindividual variability
A
differences that exist between ppl
24
Q
define intraindividual variability
A
differences that emerge in one person over time or under different circumstances
25
define distribution of scores
a set of test scores
26
define variance
statistical way of quantifying variability of individual differences
27
define covariance
way of quantifying covariability aka the connection between variability in one set of scores and variability in another set of scores
28
what is the mean?
| definition, main concept, symbol, equation
- the average persons test score
- concept: central tendency: which single score best represents the set of scores
- x
- equation: sum of all numbers / N
29
how can you calculate the differences among peoples test scores? what is the concept behind this?
concept: variability: to what degree do the scores differ from each other
2 statistics:
1. variance (s squared)
2. standard deviation (s)
30
what are the equations for variance & standard deviation?
variance = sum of all (x's - x mean) ^2/ N
standard dev = square root of variance
31
what is the skew?
| explanation & equation
- shape of a distribution of test scores
- to what degree is the set of scores evenly / symmetrically distributed around the mean?
- perfectly symmetrical distribution has skew value = 0 (normal distribution)
- distributions w long right tail = positively skewed so skew value > 0
- distributions w long left tail = negatively skewed so skew values < 0
- equation : sum of all (x's - x mean) ^3 / N*s^3
32
how can you calculate the consistency between 2 sets of scores from the same ppl?
| concept & symbols
- concept: covariability: to what degree are the differences in one set of scores consistent w the differences in another set of scores?
2 stats:
- covariance (c xy)
- correlation (r xy)
33
what is the equation for covariance?
sum of (x - x mean) (y - y mean) / N
34
what is the equation for correlation?
c xy / sx * sy
35
what is a normal distribution?
a symmetrical, bell shaped distribution, where most values cluster around the mean
36
what is a good way to visually represent the association between 2 variables?
scatterplot
shows both magnitude & direction
- upward trend of points (from lower left to upper right) indicates positive association
- downward trend of point (from upper left to lower right) indicates negative association
37
how do you interpret covariance?
positive covariance ? positive association so both variables from both distributions move in the same direction
negative covariance ? inverse relationship: one variable increases as the other decreases
38
what is the con of covariance?
only gives info about the direction of the relationship between the distributions/variables, not the magnitude (unlike correlation coefficients)
39
define covariance
degree of association between variability in 2 distributions of scores
40
what is the variance covariance matrix?
* organizes variances & covariances for multiple variables in a structured, symmetric format
* variances presented in diagonal cells, and covariances appear identically above and below the diagonal
41
how do you calculate the variance of a composite score?
variance of item a + variance of item b + variance of item etc + 2* correlation between scores on the two items * sd of a * sd of b * sd of etc
= the total test score variance will depend solely on the item variability and the correlation between item pairs
42
what does the variance of a composite score look like if the items are uncorrelated?
sum of the item variances s^2 of i + s^2 of j
43
what is a composite score?
summed or averaged responses to multiple test items
44
how do you calculate the covariance between 2 composite test scores?
c (composite 1 and composite 2) = c(ik) + c(il) + c(jk) + c(jl)
aka the sum of the covariances between the items from composite 1 (with items i and j) and composite 2 (with items k and l)
45
what is the mean of a binary item?
proportion (p) of positive responses
46
what is the variance of a binary item?
p(1-p) where p is the proportion of positive responses and 1-p is proportion of negative responses
47
when is the variance of a binary item highest?
when half the respondenst answer positively and the other half negatively because p = 1-p = 0.5 resulting in a max variance of 0.25
48
when is the variance of a binary item lowest?
if all respondents provide the same response (either all yes aka 1, or all no aka 0) then theres no variance so it cannot be correlated w other scores
49
what are the 2 ways we need to interpret psych test scores?
- quantitative interpretation: understanding whether a raw score is relatively high or low, requires comparing individuals score to entire distribution using means, sds, and normal distributions
- psych interpretion: the psych implications of the score
50
what does a z score mean?
expresses how far an individuals test score deviates from the mean in terms of sd units which allows you to see if a score is high/medium/low compared to others (z score of 0.5 says score is half an sd above mean)
51
how do you calculate a z score?
(individual score - mean)/sd
52
what are the advantages of z scores?
- provide context to raw scores, making them easier to interpret
- allow for comparisons across different tests and metrics
- essential in stats analyses like correlation computations
53
what are the limitations of z scores?
- may not be intuitive for everyone
- negative z scores or z scores in decimals may be confusing
54
what are converted standard scores aka standardized scores?
z scores that have been rescaled to make them easier for ppl to understand
55
how do you standardize z scores?
changing the mean and sd of the scores
1. select a new mean (Xnew) and new sd (SDnew) (for example X = 50, Sd = 10)
2. z score converted using formula T = z x (SDnew) x Xnew
56
what is the advantage of standardizing z scores?
- reframe the z score in more understandable and intuitive context w/o losing their meaning
- key to interpreting them is knowing the mean and sd of the converted distribution
57
what are percentile ranks?
indicates the percentage of scores below a specific score
ex: scoring at 85th percentile means test taker scored higher than 85% of others
58
what are the 2 methods for determining percentile ranks?
1. direct method: if you have access to all individual scores, can calculate by counting nr of scores below yours, dividing by total number of scores x 100
2. using z scores: if u only have mean and sd, first convert raw score to z score, then use standard normal distribution to estimate percentile rank (see table for the corresponding percentile to z rank)
59
what is the mean & sd of standard normal distirbution?
mean = 0
sd = 1
60
how can you normalize a test score to form a normal distribution ?
1. compute percentile ranks: transform raw test scores into percentile ranks based on the distribution fo scores
2. convert to z scores: use the percentile ranks to find corresponding z scores from the standard normal distribution
3. transform to standard scores: convert these z scores into a new scale w a desired mean and standard deviation
61
what is the point of normalizing test scores?
to transform a non normal distribution of scores into a distribution that approximates a normal (bell shaped) curve.
done when test developers believe that psych attribute being measured is normally distributed in pop but actual scores from test are not
62
what is the difference between standardization and normalization?
standardization: focuses on individual score comparison within the same distribution (you change an individual score to see how their position is relative to the mean (in sds)
normalization: adjusts entire distribution to fit a normal curve
63
Define dimensionality
the nr. of constructs a test measures
64
What is unidimensionality vs multidimensionality?
uni: test measures a single construct (test of attention)
multi: test measures ++ constructs (ex: Big 5 test)
65
What are the 3 main types of multidimensionality?
- multidimensional test w uncorrelated dimensions
- multidimensional tests w correlated dimensions (ex: Big 5)
- multidimensional tests w higher order factor (ex: WAIS IV, IQ test)
66
What is factor analysis?
statistical method to study dimensionality of a test
67
What are the 2 types of factor analysis?
- exploratory factor analysis (EFA): theres no theory about the factor structure (used in PTPR)
- confirmatory factor analysis (CFA): theres a clear theory about the factor structure (not used in PTPR)
68
What are the 4 steps fo EFA (exploratory factor analysis)?
1. correlation matrix: check where theres high correlations (2 items that have high correlations could form a factor)
2. check the eigenvalues
3. select a number of factors: use all factors before inflection point in scree plot (eigenvalue on y axis, factor number on x axis)
rotate the factors (orthogonal for uncorrelated factors, oblique for correalted, oblique is preferred) to see clearly what item goes with what factor
4. interpret the factors: see factor loading (item factor association)
69
what does an eigenvalue mean?
it shows how much variance one factor can account for in the data (so the more variance, the more important the factor/dimension)
70
what are the 2 methods to choose how many factors to select from the eigenvalues?
1. kaiser criterion: select all factors with an eigenvalue greater than 1.0 (dont use)
2. scree plot: eigenvalue on y axis and factor number on x axis, find the inflection point in the line and choose the factors above
71
what are factor loadings?
correlation between item and a factor (between -1 and +1)
helps us interpret factors (the higher the correlation the more likely the item belongs to the factor) then based on what items go with what factor you can name the factor
72
what is the goal of orthogonal & oblique rotation?
helps us interpret the factors by making a simpler structure of them (it helps you see which items go with which factor)
73
how do you do orthogonal rotation?
y axis represents factor 2
x axis represents factor 1
plot the 4 items on the graph based on how closely they are correlated to each factor (factor loading)
rotate the axis but keeping the 90degree angle between them to see how close the items are to each axis
remove items that dont clearly correlate with either factor
74
how do you do oblique rotation?
same as orthogonal, except you dont have to keep 90 degree angle
so the factors are correlated
75
what are factor scores?
calculate score of how much ppl score on the 2 or however many factors ppl have (for example language & science)
76
what are the advantages of factor scores?
- reduces the data
- has advantage over the composite/sum score
- avoids multicollinearity
77
what is the pattern matrix vs structure matrix?
the 2 tables you get when doing oblique rotation
pattern matrix: factor loadings controlled for the correlation between the factors (best interpretable)
structure matrix: factor loadings (more difficult to interpret cause includes correlation between factors)
78
what are the 3 key questions regarding test dimensionality?
- how many dimensions are reflected in the test items?
- are the dimensions correlated?
- what are the dimensions
79
The human development index (HDI) is a composite measure of human development consisting of health, education and income. It is used by the UN to measure the social and economic development per country. Are the scores on a HDI test reflective /effect indicators or formative /causal indicators, and why?
Formative/causal indicators, because the scores on health, education and income indicators make up a country’s HDI level
80
What are the 4 main differences in timing, completion, scoring, and difficulty between speeded tests and power tests?
power tests:
- not time limited
- completion of th entire test
- score by counting nr of items answered correctly
- pretty diffcult
speeded tests:
- time limited
- completion of part of the test
- scored by counting nr of items answered
- less difficult questions
81
A researcher conducts a longitudinal study measuring the language ability of a group of children. Over the past years, during various measuring occasions, the language ability of the girls tended to increase more than that of the boys. Therefore, the researcher expects that also at the current measuring occasion, the girls’ language ability has increased more than the boys’. What challenge of psychological testing might come into play here?
Observer/scorer bias
82
Tom participates in an experiment. During this experiment, he has to fill in a questionnaire about sustainable behavior. Tom doesn’t want others to think of him as an environmentally unfriendly person. How do we call it when Tom’s awareness that he is being assessed influences his answers on the questionnaire?
Participant reactivity
83
On which of the variables below, you expect the most individual differences in a sample of first-year university students Psychology ? Intelligence, age, or personality
Personality
84
Nathan conducts research into social anxiety for his master thesis. He uses a social anxiety questionnaire to assess the social anxiety levels of a student sample. The results show that participant 57 has a z score of 0.7. What does that mean?
That participant has a social anxiety test score that is 0.7 sds above the mean
85
Scores on the Wechsler Adult Intelligence Scale IV (WAIS-IV) have a mean of 100 and a standard deviation of 15. What kind of scores are these?
Standardized scores
86
In what situation do we use normalization transformation to obtain normalized scores?
When the raw test scores are not normally distributed
87
What is the relation between correlation and covariance?
a correlation is a standardized covariance
88
for which kind of test is it meaningful to compute a total test score (aggregating over all dimensions)?
multidimensional test w higher order factor
89
The Novaco Anger Scale is part of the two-piece Novaco Anger Scale-Provocation Inventory (NAS-PI; Novaco, 1994) and consists of 48 items measuring an individual’s anger experiences. The NAS has three subscales: cognitive, arousal and behavior. The test produces scores on each of the subscales, as well as a total test score.
What kind of test is the Nocavo Anger Scale?
Multidimensional test with a higher-order factor
90
What is a critique of using the eigenvalues-greater-than-one rule to determine the number of factors in factor analysis?
It likely results in an overestimation of the number of factors
91
what are the differences between factor matrix, pattern matrix, and structure matrix?
factor matrix:
- no rotation
- factors are uncorrelated
pattern matrix
- rotation
- controlling for factor correlations
structure matrix
- not controlling for factor correlations
