Traits: Measurement

Question 1

Name the methods of data collection.

Answer 1

• self report data
• informant report data
• life outcome data
• (situational) test data

Question 2

Name the 2 steps in the construction of self report questionnaires.

Answer 2

Step 1: identifying potential items
* by means of
- the rational approach
- empirical (or criterion keyed) approach
Step 2: factor analysis

Question 3

Self report is the most frequently used type of data when making inferences about personality. What are the 2 main methods of self report (s-data) collection?

Answer 3

1. Clinical interviews (unstructured and structured)
2. Questionnaires (open-ended and forced-choice)

Question 4

The forced-choice format is …… scored, whereas the open-ended response format is ……. scored

Answer 4

Forced choice is objectively scored.
Open- ended is Subjectively scored,

Question 5

Open-ended self report questionnaires are used extensively in what type of research? Give an example of why?

Answer 5

Cross- cultural research
In collectivist countries, participants are more likely to respond according to social roles and responsibilities.
In individualistic countries, participants are more likely to respond in terms of personal attributes.

Question 6

What is the difference between personality questionnaires/personality inventories & adjective checklists?

Answer 6

• Personality questionnaires tend to ask respondents to rate themselves on the behaviours, thoughts and feelings that are believed to define the trait of interest.
• adjective checklists ask respondents to rate themselves on the trait descriptive adjective itself (e.g. sociable, lively, active, tense, shy ..)

Question 7

What are the strengths of self-report data?

Answer 7

• Individuals have access to information about themselves that is inaccessible to anyone else. You are the worlds foremost expert on yourself. You are the only one who knows how you behave in all of life’s different situations. You are the only one who has access to your inner mental thoughts, fears, fantasies and feelings.
• Objectively scored questionnaires or adjective checklists provide a quick, cost effective way of collecting large amounts of data.

Question 8

What are the limitations of self-report data?

Answer 8

Respondents are not always willing or able to give accurate self-reports. There are different reasons why respondents give inaccurate information and these different reasons introduce unsystematic error and systemic error into your measurement of personality.

Question 9

What causes unsystematic error when dealing with self-report data?

Answer 9

• errors in self-reporting might be be brought about by factors like boredom, indifference or carelessness when completing the questionnaire.
• respondents may also have trouble comprehending the questionnaire due to reading difficulties, the effect of drugs, neurological deficits associated with developmental level and so on.

Question 10

What causes systematic error (response sets) when dealing with self-report data?

Answer 10

• respondents may have a problem with acquiescence or non-acquiescence (i.e. the tendency to agree or disagree with questions regardless of their content) or intermediate responding (i.e, the tendency to choose the safe middle option on a Likert-type scale).
• Faking or lying (impression management) - for various reasons respondents may be motivated to conceal information or even to “fake good “ or “fake bad “ on the personality questionnaire
• social desirability (self deception) - respondents may distort their responses because they hold unrealistic favourable views of themselves.

Question 11

What is the alternate view (as opposed to a limitation) for how to treat social desirability?

Answer 11

Social desirability can on the other hand be treated as a real like personality dimension worth measuring via questionnaire. It is related to emotional stableness & conscientiousness & is not necessarily a distortion of the data.

Question 12

Informant report data like self report data is collected via interviews or questionnaires. Who are common participants in informant report data collection.

Answer 12

Family, friends, teachers, psychologists, colleagues…

Question 13

What are the strengths of informant reporting?

Answer 13

• Some personality traits, for example humour, are better mead through the reactions and impressions of informants
• for individuals with limited ability to self report it is often the most effective way of gathering personality relevant information.

Question 14

What are the three main limitations of informant report data?

Answer 14

• intimate acquaintances do not have access to the target person’s inner thoughts, fears, fantasies and feelings
• the informant make lack the required observational skills
• they may not have paid close attention
• the informant may hold bias: stereotypic, specific personal (like a mother), atypical observational bias (the atypical tends to stick in people’s minds and we are not interested in the atypical)

Question 15

What is life outcome data?

Answer 15

Life outcome data is personal history information that is, in principle, publicly verifiable - the “residue” of personality (e.g., employment record, marital status, number of dependents, years at current residence, number of speeding fines and so on).

Question 16

What personality checklist is supplemented with Life outcome data and what L-data is used?

Answer 16

Psychopathy checklist - revised: a semi structured interview (S-data) is supplemented by life outcome data including police reports, criminal reports, court transcripts, medical snd psychological reports, work records..

Question 17

What is the main strength of life outcome data?

Answer 17

Information in archival records is typically accurate and not prone to the potential bias of self-report data and informant-report data.

Question 18

What is the main limitation of life outcome data and what action is normally taken to counteract this limitation?

Answer 18

Life outcomes are determined by a multitude of causal factors such as sex, ethnicity, age, educational opportunities, socio-economic status, religious orientation.
- as a consequence, life outcome data tells us very little about personality. for this reason, life outcome data is seldom used on its own to make inferences about personality. It is almost always used in conjunction with self-report and / or informant-report data.

Question 19

What is (situational) test data (T-data)?

Answer 19

Participants are either placed or found in a “testing situation “ and their behaviour is directly observed.

Question 20

Describe the 2 general types of test situation,

Answer 20

Naturalistic test situation - in which the behaviour of interest is observed in the environment in which it naturally occurs.

Artificial test situation - specifically designed to increase the likelihood of the target behaviour occurring.

Question 21

Describe the 2 main strengths of (situational) test data.

Answer 21

1. The main advantage of test-data is that the psychologist is collecting data based on direct observation rather than relying on self-report or informant-report - both of which are potentially biased.
2. Typically allows for precise objective and quantifiable measurements of variables.

Question 22

What is the main limitation of (situational) test-data?

Answer 22

Uncertain interpretation - whether it be an instance of social behaviour; psychological behaviour or the response to a Rorschach ink blot, the behaviour being monitored in the test situation does not interpret itself.
- take the bobo doll (Bandurra) experiment as an example..

Question 23

What are the 2 different strategies available for the first step in the construction of personality questionnaires ?

Answer 23

Fist step is identifying potential items

• the rational approach
• the empirical or criterion-keyed approach

Question 24

If you take the rational approach for identifying potential items, the method includes …?

Answer 24

• attempt to identify items that appear (on common sense grounds) to tap the construct of interest. For example, “I often worry about things that turn out to be unimportant “ is a rationally inspired item that would, on the face of it, appear to tap the personality construct of anxiety.

Question 25

The empirical or criterion-keyed approach to identifying potential item begins with the test developer identifying ….

Answer 25

• a criterion group extreme on the trait or characteristics of interest. - often a clinical group (e.g. individuals with anxiety disorder)
• a control group that is representative of the general population

Question 26

Adopting the criterion-keyed approach for identifying potential test items, the test developer first identifies the control group and criterion group. What does the test developer do next?

Answer 26

Devises a broad range of questions that might distinguish between these two groups.
- items may be suggested on rational or theoretical grounds but neither is by no means necessary as inspiration may come from reviews of journals and books, interviews with or observation of the criterion group, discussions with experts etc.

Question 27

The test developer administers the potential items to the criterion group and control group. How then does the test developer decide upon which items to retain?

Answer 27

The test developer retains the items that are answered differently by the two groups, to a statistically significant degree.

Question 28

What does it mean to cross-validate when referring to the identification of potential items using the criterion-keyed approach?

Answer 28

Determined from the initial examination of the criterion group and control group, the retained items are administered to a second sample of participants. These items are cross-validated by empirically distinguishing between the new criterion group and new control group.

Question 29

The 2nd step in the construction of self-report questionnaire is factor analysis. How is factor analysis administered and what is the main purpose?

Answer 29

• once items have been formulated using the rational or empirical approach, the next step is to administer the pool of items to a large group of respondents. Factor analysis is then conducted on the responses to these items
• the analysis is based on calculating correlation coefficients between each item and every other item. Items that do not correlate with other items are dropped from the pool of items. Items that correlate highly with each other are retained. (Not discussed here but it allows for the identification of factors / clusters of items that tap into different dimensions of the construct of interest)

Question 30

In general there are 2 standards that are used to evaluate the psychometric integrity of personality questionnaires. These 2 statistical requirements are?

Answer 30

• reliability
• validity

Question 31

Reliability refers to precision in measurement, yet a reliable measure is not necessarily valid. Give an example

Answer 31

Take a scale that measures weight. If the scale is out by 5kg it is still a reliable measurement as the scale will always vary precisely by 5kg. It is not valid however ad it is not an accurate measurement.

Question 32

What is test-retest reliability?

Answer 32

Is measured by calculating the correlation between scores obtained by the same persons on an identical personality questionnaire that is administered on two separate occasions (usually about 1 month apart).
- the test-retest reliability coefficient indicates the extent to which the two sets of measures generalise over the two different occasions.
- because personality is defined in terms of consistency in conduct the two sets of measurement should be highly correlated.

Question 33

We can reasonably expect scores to change over long periods of time, even for relatively stable constructs of personality, due to what types of factors?

Answer 33

Physical growth and development, learning, changes in health and living circumstances etc.

Question 34

When the interval between test and retest is greater than 6 months, the correlation coefficient is typically referred to as?

Answer 34

Stability coefficient instead of reliability coefficient

Question 35

The reliability coefficient represents the reliability of our questionnaire. When the interim between test and retest is greater than 6 months we are no longer interested in the reliability of the questionnaire but are looking at the stability coefficient which represents?

Answer 35

The stability of the personality construct itself over long periods of time.

Question 36

Reliability: Internal consistency is?

Answer 36

It estimates the degree to which items on a questionnaire jointly measure the same underlying construct.

Question 37

Reliability: the two most common examples of internal consistency are?

Answer 37

• split half reliability: correlating one half of the test with the other half of the test
• cronbach’s alpha: the average of all possible split-halves

Question 38

What is inter-rater reliability ?

Answer 38

It is the degree of agreement or consistency between two or more raters or scores.

Question 39

With questionnaires, when is inter-rater reliability relevant ?

Answer 39

It becomes relevant when the questionnaire uses an open-ended response format (scoring is subjective)

Question 40

What is indicated when there exists a high correlation of inter-rater reliability?

Answer 40

A high correlation for inter-rater reliability means that different scorers are interpreting participant responses in a similar manner (suggests a high degree of precision between the different scorers)

Question 41

Define validity.

Answer 41

The extent to which a test measures what it purports to measure.

Question 42

Name the different ways available for establishing validity.

Answer 42

• face validity
• criterion validity
• construct validity
  
  • convergent and discriminant validity

Question 43

What is meant by the term face validity?

Answer 43

Face validity is inferred when “at face value” a personality questionnaire appears to measure what it claims to measure.

Question 44

Face validity is not empirical evidence of validity. Explain why.

Answer 44

Face validity is simply the appearance of validity - and appearances can be deceptive.

Question 45

Questionnaires that have had potential items generated by means of …… are more likely to contain items with low face validity.

Answer 45

Empirical or criterion-keyed approach to construction.

Question 46

Why is face validity desirable?

Answer 46

While not an indicator of psychometric soundness it may still be desirable as an exercise in public relations.
Plus, respondents are potentially more likely to respond accurately to items high in face validity.

Question 47

Give an example of an item high in face validity and another low in face validity for the measurement of impulsivity.

Answer 47

High face validity: sometimes I do things on impulse that i later regret (rational approach)

Low face validity: do you like tall women? (Empirical / criterion-keyed approach)

Question 48

Describe criterion validity (predictive validity).

Answer 48

Criterion validity requires that a relationship be established between scores on the questionnaire and some criterion external to the questionnaire.

Question 49

Give an example of criterion validity.

Answer 49

Pilot trainees take a questionnaire designed to measure consciousness. At the end of training, evaluations are completed with trainers reporting actual levels of consciousness. The correlation between the 2 forms the validity coefficient.

Question 50

Describe construct validity.

Answer 50

Refers to whether or not the personality questionnaire measures the unobserved theoretical construct that it purports to measure.
- construct validity is founded on the total web of empirical data and theoretical analysis which build up around a construct - a nomological network (Cronbach & Meehl, 1955)

Question 51

Construct validity subsumes ……….

Answer 51

all types of validity evidence, including criterion validity.

Question 52

What procedures are commonly associated with construct validity?

Answer 52

Convergent validity and discriminant validity.

Question 53

How is convergent validity demonstrated?

Answer 53

Demonstrated when scores on a personality questionnaire correlate with scores on another personality test designed to assess the same (or a similar) construct.

Question 54

How is discriminant validity demonstrated?

Answer 54

Demonstrated when scores on a personality questionnaire show little or no correlation with scores on a measure of a theoretically different construct.