Psychological Measurement

Question 1

Reliability

Answer 1

Reliability refers to the consistency of a measure. Psychologists consider three types of consistency: over time (test-retest reliability), across items (internal consistency) and across different researchers (inter-rater reliability)

Question 2

Validity

Answer 2

Validity is the extent to which the scores from a measure represent the variable they are intended to.

Question 3

What is measurement?

Answer 3

Measurement is the assignment of scores to individuals so that the scores represent some characteristic of the individuals.

Question 4

Psychological measurement is often referred to as

Answer 4

psychometrics

Question 5

What are psychological constructs?

Answer 5

We cannot accurately assess people’s level of intelligence by looking at them, and we certainly cannot put their self-esteem on a bathroom scale. These kinds of variables are called constructs (pronounced CON-structs) and include personality traits (e.g., extraversion), emotional states (e.g., fear), attitudes (e.g., toward taxes), and abilities (e.g., athleticism).

Question 6

The conceptual definition of a psychological construct…

Answer 6

The conceptual definition of a psychological construct describes the behaviors and internal processes that make up that construct, along with how it relates to other variables.

Question 7

An operational definition…

Answer 7

An operational definition is a definition of a variable in terms of precisely how it is to be measured.

Question 8

Behavioural measures

Answer 8

Behavioral measures are those in which some other aspect of participants’ behavior is observed and recorded. This is an extremely broad category that includes the observation of people’s behavior both in highly structured laboratory tasks and in more natural settings.

Question 9

Physiological measures

Answer 9

physiological measures are those that involve recording any of a wide variety of physiological processes, including heart rate and blood pressure, galvanic skin response, hormone levels, and electrical activity and blood flow in the brain.

Question 10

Converging operations

Answer 10

When psychologists use multiple operational definitions of the same construct—either within a study or across studies—they are using converging operations.

Question 11

Levels of measurement

Answer 11

levels of measurement (which he called “scales of measurement”) correspond to four types of information that can be communicated by a set of scores, and the statistical procedures that can be used with the information.

Question 12

Nominal level of measurement

Answer 12

The nominal level of measurement is used for categorical variables and involves assigning scores that are category labels. Category labels communicate whether any two individuals are the same or different in terms of the variable being measured. e.g. marital status

Question 13

Ordinal level of measurement

Answer 13

The ordinal level of measurement involves assigning scores so that they represent the rank order of the individuals. Ranks communicate not only whether any two individuals are the same or different in terms of the variable being measured but also whether one individual is higher or lower on that variable. e.g. researcher measuring consumers’ satisfaction, requesting participants to rate their feelings as ‘very dissatisfied’ ‘somewhat disatisfied’ ‘satisfied’

Question 14

The interval level of measurement

Answer 14

The interval level of measurement involves assigning scores using numerical scales in which intervals have the same interpretation throughout. e.g. celsius scale of measurement

Question 15

The ratio level of measurement

Answer 15

The ratio level of measurement involves assigning scores in such a way that there is a true zero point that represents the complete absence of the quantity. Height measured in meters and weight measured in kilograms are good examples.

Question 16

Test-retest reliability

Answer 16

Test-retest reliability is the extent to which this is actually the case. For example, intelligence is generally thought to be consistent across time. A person who is highly
Reliability and Validity of Measurement | 95
intelligent today will be highly intelligent next week. This means that any good measure of intelligence should produce roughly the same scores for this individual next week as it does today.

Question 17

Internal consistency

Answer 17

Another kind of reliability is internal consistency, which is the consistency of people’s responses across the items on a multiple-item measure. In general, all the items on such measures are supposed to reflect
96 | Reliability and Validity of Measurement
the same underlying construct, so people’s scores on those items should be correlated with each other.

Question 18

Split-half correlation

Answer 18

Like test-retest reliability, internal consistency can only be assessed by collecting and analyzing data. One approach is to look at a split-half correlation. This involves splitting the items into two sets, such as the first and second halves of the items or the even- and odd-numbered items. Then a score is computed for each set of items, and the relationship between the two sets of scores is examined.

Question 19

Inter-rater reliability

Answer 19

Inter- rater reliability is the extent to which different observers are consistent in their judgments. For example, if you were interested in measuring university students’ social skills, you could make video recordings of them as they interacted with another student whom they are meeting for the first time. Then you could have two or more observers watch the videos and rate each student’s level of social skills.

Question 20

Face validity

Answer 20

Face validity is the extent to which a measurement method appears “on its face” to measure the construct 98 | Reliability and Validity of Measurement
of interest.

Question 21

Content validity

Answer 21

Content validity is the extent to which a measure “covers” the construct of interest. For example, if a researcher conceptually defines test anxiety as involving both sympathetic nervous system activation (leading to nervous feelings) and negative thoughts, then his measure of test anxiety should include items about both nervous feelings and negative thoughts.

Question 22

Criterion validity

Answer 22

Criterion validity is the extent to which people’s scores on a measure are correlated with other variables (known as criteria) that one would expect them to be correlated with. For example, people’s scores on a new measure of test anxiety should be negatively correlated with their performance on an important school exam.

Question 23

concurrent validity

Answer 23

When the criterion is measured at the same time as the construct, criterion validity is referred to as concurrent validity

Question 24

predictive validity

Answer 24

when the criterion is measured at some point in the future (after the construct has been measured), it is referred to as predictive validity (because scores on the measure have “predicted” a future outcome).

Question 25

convergent validity

Answer 25

one would expect new measures of test anxiety or physical risk taking to be positively correlated with existing established measures of the same constructs. This is known as convergent validity.

Question 26

Discriminant validity

Answer 26

Discriminant validity, on the other hand, is the extent to which scores on a measure are not correlated with measures of variables that are conceptually distinct. For example, self-esteem is a general attitude toward the self that is fairly stable over time. It is not the same as mood, which is how good or bad one happens to be feeling right now. So people’s scores on a new measure of self-esteem should not be very highly correlated with their moods. If the new measure of self-esteem were highly correlated with a measure of mood, it could be argued that the new measure is not really measuring self-esteem; it is measuring mood instead.

Question 27

Measurement is the

Answer 27

assignment of scores to individuals so that the scores represent some characteristic of the individuals. Psychological measurement can be achieved in a wide variety of ways, including self- report, behavioral, and physiological measures.

Question 28

Psychological constructs

Answer 28

such as intelligence, self-esteem, and depression are variables that are not directly observable because they represent behavioral tendencies or complex patterns of behavior and internal processes. An important goal of scientific research is to conceptually define psychological constructs in ways that accurately describe them.

Question 29

For any conceptual definition of a construct

Answer 29

there will be many different operational definitions or ways of measuring it. The use of multiple operational definitions, or converging operations, is a common strategy in psychological research.

Question 30

Variables can be measured at four different levels—

Answer 30

nominal, ordinal, interval, and ratio—that communicate increasing amounts of quantitative information. The level of measurement affects the kinds of statistics you can use and conclusions you can draw from your data

Question 31

Psychological researchers do not simply assume that their measures work.

Answer 31

Instead, they conduct research to show that they work. If they cannot show that they work, they stop using them.

Question 32

There are two distinct criteria by which researchers evaluate their measures:

Answer 32

reliability and validity. Reliability is consistency across time (test-retest reliability), across items (internal consistency), and across researchers (interrater reliability). Validity is the extent to which the scores actually represent the variable they are intended to.

Question 33

Validity is a judgment based on various types of evidence.

Answer 33

The relevant evidence includes the measure’s reliability, whether it covers the construct of interest, and whether the scores it produces are correlated with other variables they are expected to be correlated with and not correlated with variables that are conceptually distinct.

Question 34

Good measurement begins with

Answer 34

a clear conceptual definition of the construct to be measured. This is accomplished both by clear and detailed thinking and by a review of the research literature.

Question 35

Do you use an existing measure or create a new measure?

Answer 35

You often have the option of using an existing measure or creating a new measure. You should make this decision based on the availability of existing measures and their adequacy for your purposes.

Question 36

What steps can be taken in creating new measures and implementing new measures?

Answer 36

Several simple steps can be taken in creating new measures and in implementing both existing and new measures that can help maximize reliability and validity.

Question 37

What do you do once you have used a measure?

Answer 37

Once you have used a measure, you should reevaluate its reliability and validity based on your new data. Remember that the assessment of reliability and validity is an ongoing process.