Measurement and scaling – fundamentals, comparative and non-comparative scaling - VLE 7

Question 1

What does measurement mean?

Answer 1

Measurement means assigning numbers or other symbols to characteristics of objects according to certain pre-specified rules. There must be a one-to-one correspondence between the numbers and the characteristics being measured. The rules for assigning numbers should be standardised and applied uniformly. The rules must not change over objects or time.

Measurement means assigning numbers or other symbols to characteristics of objects according to certain pre-specified rules. We measure not the object but some characteristic of it. Therefore, we do not measure consumers, only their perceptions, attitudes, preferences or other relevant characteristics. In market research, measurement and the assigning of numbers to characteristics of consumers is done for one of two reasons. First, numbers permit statistical analysis of the resulting data. Second, numbers facilitate a universal communication of measurement rules and results.

Question 2

What are the rules of measurement?

Answer 2

There must be a one-to-one correspondence between the numbers and the characteristics being measured.

The rules for assigning numbers should be standardised and applied uniformly.

The rules must not change over objects or time.

Question 3

What does scaling mean?

Answer 3

Scaling involves creating a continuum on which measured objects are located. For example, consider an attitude scale from 1 to 100. Each participant is assigned a number from 1 to 100, with 1 =
extremely unfavourable, and 100 =
extremely favourable. Measurement is the actual assignment of a number from 1 to 100 to each participant. Scaling is the process of placing the participants on a continuum, for example with respect to their attitude toward Formula One racing.

Question 4

What is a categorical/nominal scale?

Answer 4

For a (categorical) nominal scale, the numbers serve only as labels or tags for identifying and classifying objects. When used for identification, there is a strict one-to-one correspondence between the numbers and the objects. The numbers do not reflect the amount of the characteristic possessed by the objects. The only permissible operation on the numbers in a nominal scale is counting. Only a limited number of statistics, all of which are based on frequency counts, are permissible such as percentages and the mode.

Question 6

What is an ordinal scale?

Answer 6

An ordinal scale is a ranking scale in which numbers are assigned to objects to indicate the relative extent to which the objects possess some characteristic. It is possible to determine whether an object has more or less of a characteristic than some other object, but not how much more or less. Any series of numbers can be assigned which preserves the ordered relationships between the objects. In addition to the counting operation allowable for nominal scale data, ordinal scales permit the use of statistics based on centiles such as percentiles, quartiles and the median.

Question 7

What is an interval scale?

Answer 7

For an interval scale, numerically equal distances on the scale represent equal values in the characteristic being measured. It permits comparison of the differences between objects. The location of the zero point is not fixed. Both the zero point and the units of measurement are arbitrary. Any positive linear transformation of the form y=a+bx
will preserve the properties of the scale. It is not meaningful to take ratios of scale values. Statistical techniques which may be used include all of those which can be applied to nominal and ordinal data. In addition, the arithmetic mean, standard deviation, and other statistics commonly used in market research are applicable.

Question 8

What is a ratio scale?

Answer 8

A ratio scale possesses all the properties of the nominal, ordinal and interval scales. It has an absolute zero point. It is meaningful to compute ratios of scale values. Only proportionate transformations of the form y=bx, where b
is a positive constant, are allowed. All statistical techniques can be applied to ratio data.

Question 9

What is the difference between nominal and ordinal scales?

Answer 9

In nominal scales, the numbers serve only as labels or tags for identifying and classifying objects, while in an ordinal scale the numbers are used as a ranking device.

Question 10

What are the advantages of a ratio scale over an interval scale? Are these advantages significant?

Answer 10

The key advantage which a ratio scale has is that the origin is fixed. Therefore, it is meaningful to take ratios of scale values. Statistics such as a geometric mean, harmonic mean and coefficient of variation can be applied to analyse ratio scale data.

Question 11

What are comparative scales?

Answer 11

Comparative scales involve the direct comparison of stimulus objects. Comparative scale data must be interpreted in relative terms and have only ordinal or rank-order properties.

Question 12

What are non comparative scales?

Answer 12

In non-comparative scales, each object is scaled independently of the others in the stimulus set. The resulting data are generally assumed to be interval- or ratio-scaled.

Question 13

What are advantages/disadvantages of comparative scales?

Answer 13

Advantages:

Small differences between stimulus objects can be detected.

There are the same known reference points for all participants.

Easily understood and can be applied.

Involves fewer theoretical assumptions.

Tends to reduce halo or carryover effects from one judgement to another.

Disadvantages:

Ordinal nature of the data.

Inability to generalise beyond the stimulus objects scaled.

Question 14

What is paired comparison scaling?

Answer 14

Paired comparison scaling is the most widely-used comparative scaling technique. A participant is presented with two objects and asked to select one according to some criterion. The data obtained are ordinal in nature. With n
brands, n(n−1)/2
paired comparisons are required. Under the assumption of transitivity of preference, it is possible to convert paired comparison data to a rank order

Question 15

What is taste testing?

Answer 15

The most common method of taste testing is paired comparison. The consumer is asked to sample two different products and select the one with the most appealing taste. The test is done in private and a minimum of 1,000 responses is considered an adequate sample size.

Question 16

What could be a problem with blind taste tests?

Answer 16

A blind taste test for a soft drink, where imagery, self-perception and brand reputation are very important factors in the consumer’s purchasing decision, may not be a good indicator of performance in the marketplace. The introduction of New Coke illustrates this point. New Coke was heavily favoured in blind paired comparison taste tests, but its introduction was less than successful, because image plays a major role in the purchase of Coca-Cola.

Question 17

What is rank order scaling?

Answer 17

For rank-order scaling, participants are presented with several objects simultaneously and asked to order or rank them according to some criterion. It is possible that the participant may dislike the brand ranked 1 in an absolute sense. Furthermore, rank-order scaling also results in ordinal data. Only n−1 scaling decisions need to be made in rank-order scaling

Question 18

What is constant sum scaling?

Answer 18

For constant sum scaling, participants allocate a constant sum of units, such as 100 points, to attributes of a product to reflect their importance. If an attribute is unimportant, the participant assigns it zero points. If an attribute is twice as important as some other attribute, it receives twice as many points. The sum of all the points is 100, hence the name of the scale! Figure 12.5 of the textbook provides an example of the importance of bottled-beer attributes using a constant sum scale.

Question 19

What is a monadic scale?

Answer 19

In a non-comparative scale, participants evaluate only one object at a time, and for this reason non-comparative scales are often referred to as monadic scales.

Question 20

What are the two types of non-comparative scales?

Answer 20

Continuous

Itemised rating scales

Question 21

What is an itemised rating scale?

Answer 21

For an itemised rating scale, participants are provided with a scale which has a number or brief description associated with each category. The categories are ordered in terms of scale position, and the participants are required to select the specified category which best describes the object being rated. The commonly-used itemised rating scales are the Likert scale, semantic differential and Stapel scale.

Question 22

Describe the semantic differential scale and the Likert scale. For what purposes are these scales used?

Answer 22

A semantic differential scale is a seven-point rating scale with bipolar labels which have semantic meaning. In a typical application, participants rate objects on a series of itemised, seven-point rating scales, bounded at each end by one of two bipolar adjectives, such as ‘powerful’ or ‘weak’. The Likert scale typically has five response categories ranging from ‘strongly disagree’ to ‘strongly agree’. Participants are required to indicate a degree of agreement or disagreement with each of a series of statements related to the stimulus objects.

These scales are used to measure the strength of feeling about the individual constructs or components of marketing phenomena such as brand, product and company images, feelings about advertising and promotion strategies, new product development studies and in a variety of other applications.

Question 23

What decisions should be considered when dealing with itemised rating scales?

Answer 23

The number of scale categories to use. Although there is no single, optimal number, traditional guidelines suggest that there should be between five and nine categories.

Balanced versus unbalanced scale. In general, the scale should be balanced to obtain the most objective data. Figure 12.10 of the textbook provides an example of balanced and unbalanced scales.

Odd or even number of categories. If a neutral or indifferent scale response is possible from at least some of the participants, an odd number of categories should be used.

Forced versus unforced choice. In situations where the participants are expected to have no opinion, the accuracy of the data may be improved by a non-forced scale.

Nature and degree of verbal description. An argument can be made for labelling all or many scale categories. The category descriptions should be located as close to the response categories as possible.

Physical form of the scale. A number of options should be tried and the best one selected. Figure 12.11 of the textbook provides an example of different rating scale configurations. Figure 12.12 of the textbook provides examples of some unique rating scale configurations.

Question 24

How should you select a particular scaling technique?

Answer 24

There are two perspectives to consider in selecting a scaling technique. First, the researcher must consider what will work with a target participant. Will they understand what is required and take the time to reflect on the issues and answer honestly? Second, the researcher should consider which statistical techniques would yield the best understanding of an issue, given the research objectives set.

When choosing a particular scaling technique, an attempt should be made to use the one which will yield the highest level of information feasible in the given situation. In many situations, in order to satisfy the above two perspectives, it may be desirable to use more than one scaling technique or to obtain additional measures using procedures other than the conventional scaling techniques.

Question 25

‘A brand could receive the highest median rank on a rank-order scale of all the brands considered and still have poor sales.’ Discuss.

Answer 25

Pertinent to this discussion is the issue of what is being measured. Products can be perceived very positively, yet economic or marketing considerations may negatively impact sales. If we are measuring attitudes toward the brand, it can be very highly rated, however a high price may be charged which eliminates most buyers – for example, in the car market, consider a Porsche. Attitudes and intentions do not perfectly correlate with behaviour because there are additional factors affecting behaviour. Another issue is that none of the brands considered may be desirable. Hence, even the top-ranked brand may be undesirable.

Question 26

How does the true score model show the accuracy of measurement?

Answer 26

XO=XT+XS+XR

where:

XO=
the observed score or measurement

XT=
the true score of the characteristic

XS=
the systematic error

XR=
the random error.

Question 27

What could be potential sources of error in measurement?

Answer 27

other relatively stable characteristics of the individual which influence the test score, such as intelligence, social desirability and education

short-term or transient personal factors, such as health, emotions and fatigue

situational factors, such as the presence of other people, noise and distractions

sampling of items included in the scale: addition, deletion or changes in the scale items.

lack of clarity of the scale, including the instructions or the items themselves

mechanical factors, such as poor printing, overcrowding items in the questionnaire and poor design

administration of the scale, such as differences among interviewers

analysis factors, such as differences in scoring and statistical analysis.

Question 28

What is reliability?

Answer 28

Reliability can be defined as the extent to which measures are free from random error, XR. If XR=0, the measure is perfectly reliable.

Question 29

What kinds of reliabilities can you name?

Answer 29

In-test
Alternative forms
Internal consistency
Split-half

In test-retest reliability, participants are administered identical sets of scale items at two different times and the degree of similarity between the two measurements is determined. In alternative-forms reliability, two equivalent forms of the scale are constructed and the same participants are measured at two different times, with a different form being used each time.

Internal consistency reliability determines the extent to which different parts of a summated scale are consistent in what they indicate about the characteristic being measured. In split-half reliability, the items on the scale are divided into two halves and the resulting half scores are correlated. Cronbach’s alpha, or the coefficient alpha, is the average of all possible split-half coefficients resulting from different ways of splitting the scale items. This coefficient varies from 0 to 1, and a value of 0.6 or less generally indicates unsatisfactory internal consistency reliability.

Question 30

What does validity mean?

Answer 30

The validity of a scale may be defined as the extent to which differences in observed scale scores reflect true differences among objects on the characteristic being measured, rather than systematic or random error. Perfect validity requires that there be no measurement error (XO=XT, XR=0
and XS=0).

Question 31

What kinds of validities should be considered?

Answer 31

Content validity
is a subjective but systematic evaluation of how well the content of a scale represents the measurement task at hand.

Criterion validity
reflects whether a scale performs as expected in relation to other variables selected (criterion variables) as meaningful criteria.

Concurrent validity
is assessed when the data on the scale being evaluated and on the criterion variables are collected at the same time.

Predictive validity
is concerned with how well a scale can forecast a future criterion.

Construct validity
addresses the question of which construct or characteristic the scale is, in fact, measuring. This includes convergent, discriminant and nomological validity.

Convergent validity
is the extent to which the scale correlates positively with other measurements of the same construct.

Discriminant validity
is the extent to which a measure does not correlate with other constructs from which it is supposed to differ.

Nomological validity
is the extent to which the scale correlates in theoretically predicted ways with measures of different but related constructs.

Question 32

What is the relationship between validity and reliability?

Answer 32

If a measure is perfectly valid, it is also perfectly reliable. In this case XO=XT, XR=0 and XS=0 . If a measure is unreliable, it cannot be perfectly valid, since at a minimum XO=XT+XR. Furthermore, systematic error may also be present, i.e. XS≠0. Therefore, unreliability implies invalidity. If a measure is perfectly reliable, it may or may not be perfectly valid, because systematic error may still be present (XO=XT+XS).

Reliability is a necessary, but not sufficient, condition for validity. Generalisability refers to the extent to which one can generalise from the observations at hand to a universe of generalisations.