Part 1

Question 1

What does the Greek of psycho- mean

Answer 1

Greek root (psyche) means breath, spirit, or soul

Question 2

What does the Greek of -metrics mean

Answer 2

Greek root (metron) means measure, size, or distance

Question 3

Name 6 reasons of studying measurements

Answer 3

1. minimize subjectivity of judgement: however, in psychology, a lot of the experience is subjective
2. make more precise statements
3. quantify your observations
4. Ensure reliable & valid measures, essential to sound science
5. Application: good judgements require good measurement, e.g. applying the DSM-5
6. Questionnaires are good to make dependent variables, they also help eliminate error as covariates, controls, or experimental groups

Question 4

Even if a measurement is never perfect, how can you quantify how wrong you might be?

Answer 4

we need to assess the degree of error

Question 5

What can introduce error to measuring?

Answer 5

1. measurement itself can affect error

2. both participants and researchers can introduce biases to the data

Question 6

What is the first test you should have done in your life?

Answer 6

APGAR test

Question 7

What is the apgar test?

Answer 7

it assesses your general level of functioning right after birth

infants are tested on 5 categories, with scores ranging from 0 to 2 for each one
Appearance (colour), Pulse, Grimace, Activity & Respiration

converting observations into quantifiable values (number)

Question 8

What is empirical thinking?

Answer 8

Assuming our observations are valuable

As opposed to trusting a higher truth, for example, because you may not be able to trust your senses

Question 9

Who is Francis Galton, what did he contribute

Answer 9

A, if not the, founder of psychometrics
he was perhaps a bit obsessed with observation and measurement
e.g. brush strokes needed to complete a painting, the number of times people fidget during a lecture, developed the first weather map, calculating the degree of association between any two characteristics

For us, his most critical accomplishment was the recognition of individual differences
understanding the ways in which people differ
how do we calculate how large these differences are, and what causes them

Question 10

What is the difference between traits and states

Answer 10

Trait differences
e.g. extraversion, IQ, depression, anxiety
resistant to change over time
refer to behaviour in general
often easier to measure with questionnaires

State differences
e.g. sleepiness, hunger, depression, anxiety
subject to change over short time periods
refer to behaviour in the moment (i.e. now)
a task tend to measure a state
easy to measure with tasks and questionnaire

Question 11

Name 2 different types of scaling

Answer 11

1. Stimulus-centered scaling: also known as psychophysics (mostly objective), it’s the relation of physical, directly measurable, stimuli to perception
   e. g. how little sound is physically possible to hear is objective stimuli, but an subjective experience

2. subject-centered scaling: estimating the subjective presence, absence or degree of a construct
most common in psychology, topic of this class
we attend to reduce/manage the subjectivity and minimize error from that source

Question 12

How can questions be turned into measurements

Answer 12

The answers must be converted into numbers depending on the thing we are trying to measure
e.g. False = 0 Not sure = 1 True = 2

Question 13

What is a benefit of including the numerical value to the questionnaires’ answers

Answer 13

0 = Not at all, 1 = A little, 2 =Moderate, 3 = Severe

some people like to include the scale to give a better idea while evaluating which response they would give

Question 14

Name the 4 types of scales

Answer 14

1. Nominal
2. Ordinal
3. Interval
4. Ratio

Question 15

Why and how do we introduce numbers in nominal scales

Answer 15

Numbers are assigned as levels only
The numbers could just as easily be words, and using words instead would have no impact on how useful the measurement is to us
e..g when measuring sex, we could assign a value of 0 to women and 1 to men

this would not mean men are better than women, the numeric increase has no inherent value
for analysis purposes, it’s sometimes necessary to code sex this way

Question 16

Describe the ordinal scale

Answer 16

numbers are not just labels, they also serve to rank individuals. Now we must be using numbers, and in a meaningful way
e.g. an ordinal scaling of height might be assign 1 to the smallest participant, 2 to the next smallest participant, and so on

if we coded women as 0, and men as 1 and claim it’s an ordinal scale, then we are saying men are better in some way

Question 17

Describe the interval scale

Answer 17

Numbers are labels, and reflect rank, and tell us exactly how much more of something we have now. With an ordinal scale the practical difference between 1 and 2 might be smaller or larger than the practical difference between 2 and 3
e.g. if we go from 28 to 18 celsius we have lost 10 celsius and this is the same degree of change as going from 28 to 38 celsius

true even if from a psychological perspective it’s not perceptually the same

Question 18

Describe the ratio scale

Answer 18

numbers are labels. and reflect rank, and tell us exactly how much more of something we have, and a score of zero is the smallest possible number

ratio scaled numbers are essentially the same as interval scaled numbers, except that they can’t be negative. Aside from possible interpretation differences, for psychology, ratio numbers are functionally the same as interval numbers
in questionnaires, there usually is never ‘nothing’ as a possibility
starting a scale at 1 instead of 0 can indicate you are acknowledging it is not used as a ratio scale

Question 19

What do we expect from our distribution of data

Answer 19

we expect a normal distribution

Question 20

Describe the two different use of the word ‘standard’ in psychometrics

Answer 20

Common units: to compare numbers measure with different scaling methods, we standardize to a common (Z) unit

Common understanding: to compare results across participants, we norm-reference them
“good” performance is a relative term
a raw score on a measurement rarely has much interpretative value on its own

Question 21

How do we develop good standard scores

Answer 21

1. Provide a new measure to a very large sample. It is necessary to achieve a normal distribution of scores, this ensures we have properly capture the range of natural variability.
2. Verify the data represent the full range of scores
   e. g. if a scale is from 1 to 9, someone must have picked 1 and another 9 to make sure the full range can be captures
3. Determine the distribution of the scores
4. Break sores down into psychologically meaningful groups

Question 22

Explain the two meanings of the word ‘scale’

Answer 22

1. describe different measurement systems (e.g. subject-centered scaling, or interval number scales)
2. questionnaires or survey (e.g. boredom proneness scale). Using scale in this scenario imply we are measuring a unitary construct (as opposed to a combination of traits such as the Big5)

Question 23

Name the 4 steps to developing a questionnaire

Answer 23

1. Construct Identification
   What is the ‘thing’ you’re interested in really about?
   How do you measure it in its entirety
2. Make items: literature review
   Review existing literature, and consult experts
   Make some questions, then make a lot more
3. Pick a response format
   What response format is most effective?
   Use just one format for all questions?
4. Pilot test
   Find out which items work best
   Reduce items, and repeat

Question 24

Name some challenges of making items for a questionnaire

Answer 24

You cannot be certain you are making good items. The readings tell you about the problems you can encounter, but they don’t give you a set of easily-followed rules to avoid problems.

Plan as much as you can, and hope for the best
that said, there are some good guidelines you can try to follow

Making items involves several steps

1. one of the things you had to decide in Step 1 was how to weight each aspect of your construct (if it’s not a simple one); now you need a sufficient number of items to produce those weights
2. puzzling over how to word your questions, and the potential consequences
3. deciding whether to reverse-word some items; how many?
4. deciding whether to have redundant questions; how many? early on yes, this way you can make sure which wording is best, but eventually no to reduce the amount of items

Question 25

How many items should you include in your questionnaire?

Answer 25

early in the process, more is always better. Later in the process, more is not always better

Aim for the Goldilocks zone; balancing construct accuracy and error correction with time. So, how many items you need depends heavily on your construct definition
very simple: 4-8 items
very complex: 40-60 items
usually never need to go over a hundred, but you might

Question 26

Why or why not use reverse-wording items

Answer 26

sometimes we need or want to word our items in a way that reflects the opposite of our chosen construct
to make the scores reflect the construct, we simply reverse-score.

1. can be useful for breaking participants’ habitual responses
   a, identify who does not pay enough attention
   b. can prevent bias responsive
2. can be problematic because the ‘opposite’ of our construct might not be easily defined
   a. it can introduce bias responding by the inability to produce exact opposite items

Question 27

Why or why not use redundant items, give good and bad examples

Answer 27

1. use it to strengthen a scale
   a. rephrasing items in similar ways bring out their common cause, reducing idiosyncrasies in how a particular wording may reflect the construct
2. avoid using bad redundancy
   a. these rephrasing are trivially different, different but not usefully different, items may simply have a small difference in grammatical structure

Bad examples
a really important thing is my child’s success
the really important thing is my child’s success
not a bad idea to include both in pilot testing, but remove one at the end

Good examples
I will do almost anything to ensure my child’s success
no sacrifice is too great if it helps my child succeed

Question 28

Describe what are dichotomous and semantic differential response formats

Answer 28

1. Dichotomous, two choices:
   - more questions always make a questionnaire better
   a. true
   b. false
2. Semantic Differential:
   I feel psychometrics is
   Boring 1 2 3 4 5 Interesting
   Unnecessary 1 2 3 4 5 Necessary

Adjectives should be used on either end of the semantic differential format and discrete choices (1,2,3 as opposed to 1.5)

Question 29

Describe the Thurstone Scale

Answer 29

Construct: Parental aspirations for children’s career/educational success
Based on Thursdone’s law of comparative judgment, which attempts to produce proper interval-level data
Equally calibrated difference between each item
Must agree to one, and disagree to three statements for each item
however, people could agree with more than one statement leaving no room to analyze the data
Makes a really long questionnaire since each item has a lot of semi-long statements

Question 30

Describe the Guttman Scale

Answer 30

Based on Thurston Scales
Tried to make it a little easier to construct questions
Still the same drawback of a lot of questions/long questionnaires
Could agree with 2 statements and disagree with 2 (as opposed to 1/3)
should have one change of response set
e.g. should agree to all item until you start disagreeing
or should disagree to all item until you start to agree
the score will be attributed to where the change occurred
if more than one change, it’s possible to get uninterpretable responses from participants

Question 31

Describe analog scales

Answer 31

A recent and much more common, and increasingly so, option is the analog scale
Not at all 0 ———————– 100 Very much
Unsatisfactory 0 1 2 3 4 5 Outstanding

The answer may be anywhere on the line, not strictly limited to the numbers (e.g. could be 1.59)
High level of detail, doesn’t have to start at zero, not need for numbers, but allowed
Didn’t used to be too common because it was not offered digitally. On paper, it can get tricky and long to measure it out and score it

Question 32

Describe the Likert- and Likert-type Scale

Answer 32

Likert (1932) Scale:
Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree
Must be 5 options, must be exactly those labels for agreement with the statement
People can remain neutral

Likert-type Scale:
Almost never, very infrequently, somewhat infrequently, somewhat frequently, almost almost
Doesn’t have to be exactly 5 options, must specify what each number represent, can be frequency, agreement, or others
Does a good job balance how easy it is to write the questions, easiness of responding (people don’t have to read all of the possible answers), can potentially have more questions, most popular these days

Question 33

Name 3 pitfall to avoid in creating items

Answer 33

1. Don’t use double negatives
2. Don’t use double-barreled questions
3. Don’t use leading questions

Question 34

Name two ways double negatives may impact your questionnaires

Answer 34

1. Double negatives in the question (e.g. I might not never vote for the conversative party | True - False)
2. Negative in the question AND the answer (e.g. Drivers should never use cell phones | Always 1 2 3 4 5 Never)

Question 35

How can you fix double-barreled questions

Answer 35

Make two questions

Question 36

How does leading questions impact your participants

Answer 36

these question bias the responses, reducing variability and increasing systematic error

does not lead to a normal distribution, violating assumptions

both confusion and bias are bad for validity

Question 37

Name 5 response sets, how to recognize them and/or fix it

Answer 37

yea-saying: agreement with everything (or always answering in a positive way). Can be recognized through reverse-worded questions.

nae-saying: saying no to everything, but much less common. Can be recognized through reverse-worded questions.

fence-sitting: sitting in the middle, will not take a position on anything. More common than nae-saying, less than yea-saying. Solution is to remove the middle option and force them to pick a side.

social desirability: answering in a way that makes you look like a better person (but you’re not being perfectly truthful). can be confused with yea-saying, but sometimes the socially desirable answer can be ‘no’.
some questions have been developed to identity this

malingering: answering in a way that makes you look like a bad person (but you’re not) e.g. in a legal situation. some questions have been developed to identity this (e.g. malingering scales) e.g. someone with schizophrenia would say no, someone who doesn’t would think someone with schizophrenia would say yes

Question 38

Why should we balance our scales

Answer 38

When we use Likert and Likert-type response formats, we assume (incorrectly) that we have equal intervals
It’s very important to us that this assumption is at least approximately true

you can introduce bias if the negative/positive is positioned on the right vs the left. Numbers should match what we expect to be negative to positive

Question 39

Should you play around where positive, negative, high and low values are found on the answers?

Answer 39

No, it’s generally best to put ‘low’ -value responses (e.g. disagree, never, dissatisfied, etc.) on the left even if there is a left-side response bias
this is because our brain is used to processing information in a certain way , going against that can produce more error
snark effect: people have an inherent spatial arrangement of numbers in their brain

Question 40

Name some costs of poor measurement

Answer 40

using a poor measure is sometimes worse than no measure at all

validity: almost by definition, it’s going to have poor validity

correlations: a poor measure will make it hard to find the expected degree of association between two things, leading to potential mistakes
if too much error = underestimate the association
if too much systematic bias = overestimate the association

validity again: a poor measure probably doesn’t capture the full construct, leading to potentially inaccurate conclusions about why variables are correlated

Question 41

What did Jonathan et al. found about the Boredom Proneness Scale

Answer 41

By undoing the reverse-worded questions, there were no longer 2 distinct sub constructs present in the data of the Boredom Proneness Scale.

In this scenario, this is a big source of error than trying to combat yea-saying since it causes error for everyone, as opposed to a few single cases
it also showed that it could be less than 28 questions since it is a single not too complex construct. We can measure it just as well with 8 questions

Question 42

Name two ways of interpreting the results of a questionnaire

Answer 42

Total: usually the sum of the individual item responses
Mean: the arithmetic average of the individual item responses

Question 43

When is it appropriate to use total scores vs mean scores

Answer 43

If not everybody answer every question, then the mean would be the superior choice

If everybody answers every questions, than the total and the mean will functionally have the same purpose

Question 44

What is the goal of ‘norm referencing’

Answer 44

norm-referencing: is a way to improve your interpretation of raw scores and to give better feedback on performance

Question 45

Name the steps to norm referencing

Answer 45

1. examining the distribution we obtained with our data: did it reach the full range possible
2. Dividing people into groups (class-interval)
3. Create a histogram to visualize if it is a normal distribution
4. Determine the standard deviation and label groups accordingly or create percentile ranks

Question 46

Define “frequency”, “relative frequency”, “cumulative frequency” and “cumulative relative frequency”

Answer 46

frequency: how many people has a similar score (e.g. frequency of 4 on score 18 means 4 people scored 18)

relative frequency: turning frequency into a percentage

cumulative frequency: how many people up to that point

cumulative relative frequency: turning cumulative frequency into a percentage

Question 47

Why do we use percentile ranks

Answer 47

A percentile is a normative way of describing performance on a test/questionnaire

It explains how many people would score lower than you on that measure

Question 48

Describe the two ways to calculate percentile ranks

Answer 48

There are 2 ways to calculate percentiles

Within a sample: using one set of data. Minimum is 1, maximum is 99

Across samples: using multiple data sets to compare the new score to. Periodically, you update the reference data set. Minimum is 1, maximum is 100

Question 49

If you score 5th and 99th percentile ranks on a sample of 100, which person in relation to the others would you be

Answer 49

6th and the 100th person

Question 50

What type of data does percentile produce

Answer 50

Ordinal.

Percentiles aren’t equal intervals. They seem like they should give interval-level data, but they’re just pretending.

Question 51

If I score 2.63 as my final percentile ranks, what is my final percentile

Answer 51

3

Question 52

If I score 0.12 as my final percentile ranks, what is my final percentile

Answer 52

1, you cannot have 0.

Question 53

What is the highest percentile you may have

Answer 53

Within subject: 99

Between subject: 100

Question 54

What are some important considerations when talking about percentiles

Answer 54

We don’t give a score of 0 to not be overly harsh

If describing someone using a percentile score it’s important to describe them accurately
For the 60th percentile:
‘60% of people would have obtained a lower score than you’

For the 90th percentile:
‘Your score was in the top 10%’
Or, for score from 90-100, just say ‘Well above average’, especially if undesirable behavior when you don’t want to give the exact number

the extent to which you decide if you are willing to go with a precise number should depend on how trustworthy you think your percentiles are, largely dictated by sample size