week 5 - concepts in psychometric testing

Question 1

What is the “black box” problem?

Answer 1

Primary issue with assessing behavior. You cannot observe another person’s thoughts.

Question 2

What is self-report?

Answer 2

Method of data collection involves questionnaires and interviews, advantageous because only self knows thoughts, disadvantageous because of issues with honesty/reflective language.

Question 3

What is observation?

Answer 3

Method of data collection involves observing response to place/task, advantageous in that it is objective and can be done without vested interest, disadvantageous in that it is time-consuming and offers limited insight.

Question 4

What is other-report?

Answer 4

Method of data collection involves information from friend/family reporting, pros “everyday” source of information, con issues with honesty and accuracy.

Question 5

What is biological measurement?

Answer 5

Method of data collection involves tools such as fMRI and EEG, pro - objective measurement of biology, con not a direct observation of psychology.

Question 6

What are projective tests?

Answer 6

A type of psychological tool for measuring people which involves things like an inkblot test, draw a person, handwriting analysis, etc.

Question 7

What are psychometric tests?

Answer 7

A type of psychological tool for measuring people can be broken down into maximal testing (ability, iq), and typical testing (traits, personality).

Question 8

What is a common concern with projective tests?

Answer 8

Issues with falsifiability, substantial lack of evidence and validity, leads to the question can you even validate/reliably evaluate the subconscious? Can you science the subconscious?

Question 9

What is internal reliability?

Answer 9

Within a study, how often do measures agree?

Question 10

What is external reliability?

Answer 10

Replicability, across a study how often do your findings agree?

Question 11

What is conceptual replication?

Answer 11

same idea, different method, different sample/data

Question 12

What is direct replication?

Answer 12

same idea, same methods, different sample/data

Question 13

What is convergent validity?

Answer 13

The statistical relationship between two similar measures.i.e. measures of happiness and life satisfaction converge.

Question 14

What is concurrent validity?

Answer 14

The statistical relationship with a similar assessment of the same construct.i.e. a new measure of happiness is concurrent with an old one.

Question 15

What is face validity?

Answer 15

Does this have the appearance of good measurement? i.e. to what extent can overlapping circles represent community feeling.

Question 16

What is content validity?

Answer 16

The value of the included/excluded content for representing a construct.i.e. single item measures not fully representing all aspects of construct.

Question 17

Which z score is the mean response?

Answer 17

z= 0

Question 18

What z scores indicate someone is above the mean but within the normal range?

Answer 18

anything between z = 0 and z = +1

Question 19

What z scores indicate someone is below the mean but within the normal range?

Answer 19

anything between z = 0 and z = -1.

Question 20

What is an atypically high or low z score?

Answer 20

anything above z = +1 or below z = -1.

Question 21

When communicating to someone about their test scores on a test you should tell them…

Answer 21

1) What their norm-referenced score means relative to population.
2) How this understanding of norms might describe their behaviour.
3) What are the limitations and benefits of the particular assessment.

Question 22

What is the ‘normal range’ of T scores for any given test?

Answer 22

40-60

Question 23

If someone completes a test and receives a Sten score of 8.00, how would we refer to this score?

Answer 23

Notably above average.

Question 24

If someone completes a test and receives a z score of -0.91, how would we refer to this score?

Answer 24

In the normal range.

Question 25

Susan has applied for a job as a construction site manager. The company she wants to work for screens all their employees with the Big Five using a self-report test.

In particular, they are interested in Susan’s Conscientiousness score.

What does Conscientiousness measure?

Answer 25

Orderliness, organisation

Question 26

Susan has applied for a job as a construction site manager. The company she wants to work for screens all their employees with the Big Five with a self-report test. In particular, they are interested in Susan’s Conscientiousness score.

A respondent’s average response on a 1 (Strongly Disagree) to 5 (Strongly Agree) scale for Conscientiousness is retained for analysis. A higher score is indicative of more Conscientiousness-like behaviour.

The norm range suitable for Susan’s age and gender is Mean = 3.52 and SD= 0.34

Susan’s score for Conscientiousness is 4.12.

What is Susan’s score expressed as a z score?

Answer 26

1.76

Question 27

How is a z score computed?

Answer 27

a respondent’s score (x) less the mean (M), divided by the standard deviation (SD):

z= (x-M)/SD.

Question 28

How do we get a T score?

Answer 28

The T score is converted from a z score as 10z + 50.

The z score is computed as a respondent’s score (x) less the mean (M), divided by the standard deviation (SD):

z= (x-M)/SD

Question 29

We have seen that Susan’s z score for Conscientiousness is 1.76 (and therefore T= 67.65).

What does this score mean for Susan’s Conscientiousness?

Answer 29

Notably above average.

Question 30

We have seen that Susan’s z score for Conscientiousness is 1.76 (and therefore T= 67.65).

How might you describe Susan’s behaviour in general?

Answer 30

I would expect to see someone tell Susan that she is generally more likely to be organised and structured in her thought and behaviour. She is like to plan ahead and structure her life - more so than the average person.

I would expect someone to to describe her behaviour in line with higher Conscientiousness