The Theory of Measurement

Question 1

What is the lowest level of measurement?

Answer 1

Nominal

Question 2

What does nominal refer to?

Answer 2

A classification where use of numbers simply as labels for categories (e.g., 1 = male, 2 = female)

Question 3

What is the measurement of ordinal?

Answer 3

Responses are ordered or ranked in terms of magnitude and a sequential number is assigned to each. “Mild,” “moderate,” and “severe” disability might be coded 1, 2, and 3

Question 4

What is the measurement of interval?

Answer 4

Numbers are assigned to the response categories in such a way that the distance between two numbers is known, e.g. temperature in degrees Celcius.

Question 5

What is the measurement of ratio?

Answer 5

The key here is a meaningful zero point, making it possible to state that one score is, for example, twice another, e.g. weight in kg.

Question 6

Define ‘objective’

Answer 6

Involve no human judgment in the collection and processing.

Question 7

Define ‘subjective’

Answer 7

Judgments of a person.

Question 8

Define ‘single item subjective instruments’.

Answer 8

Self-reported questions with using single questions, rating or item to measure a concept.

Question 9

Define ‘battery subjective instruments’.

Answer 9

Series of self-reported questions, ratings or items used to measure a concept.

Question 10

Define ‘scale subjective instruments’.

Answer 10

Series of self-report questions, ratings or items used to measure a concept, but response categories are all in the same format and are summed and may be weighted.

Question 11

Define ‘face validity’ and give an example.

Answer 11

Subjective assessment by the investigator about whether the instrument appears. Example: question ‘Do you usually cough first thing in the morning?’ has face validity as an indicator of respiratory disease.

Question 12

What is the issue with face validity?

Answer 12

We do not want the subjects to know what we are doing and so be able to conceal things from us, e.g. assessing underlying attitudes to ethnicity.

Question 13

Define ‘content validity’.

Answer 13

The degree to which a test or assessment instrument evaluates all aspects of the topic, construct, or behaviour that it is designed to measure.

Question 14

What is the difference between face validity and content validity?

Answer 14

Face validity appears valid to the subjects, content validity appears valid to an expert.

Question 15

When does a measurement technique have criterion validity?

Answer 15

If its results are closely related to those given by some other, definitive technique, a ‘gold standard’.

Question 16

Define ‘criterion validity’.

Answer 16

Evaluates how accurately a test measures the outcome it was designed to measure.

Question 17

List two sub-types of criterion validity.

Answer 17

• Concurrent validity.
• Predictive validity.

Question 18

Define ‘concurrent validity’.

Answer 18

Testing a scale against another, measuring the same thing (at the same time) – when developing a new scale.

Question 19

Define ‘predictive validity’.

Answer 19

Whether measures can predict another key variable in the future (e.g. self-rated health is able to predict mortality even after controlling for specific health status indicators (Ilder & Benyamini, 1997).

Question 20

Define ‘construct validity’.

Answer 20

Whether the instrument is measuring the underlying concept.

Question 21

When is construct validity appropriate to use?

Answer 21

When the variable of interest cannot be observed.

Question 22

Define ‘convergent validity’.

Answer 22

Refers to how closely the new scale is related to other variables and other measures of the same construct.

Question 23

Define ‘discriminant (divergent) validity’.

Answer 23

Used to determine if a test is too similar to another test.

Question 24

Define ‘reliability’.

Answer 24

When a measure consistently produces the same results, particularly when applied to the same subjects at different time points when there is no evidence of change.

Question 25

List methods of reliability.

Answer 25

• Internal consistency
• Test-retest
• Intra-rater agreement
• Inter-rater agreement

Question 26

Why should measures be sensitive to differences in health states?

Answer 26

A measure needs to able to detect changes over time, or mean differences between groups, in the aspect of health that it is intended to measure.

Question 27

List aspects that make a health indicator good.

Answer 27

• Reliable
• Valid
• Appropriate
• Relevant
• Responsiveness
• Practical

Question 28

List the three different types of scaling.

Answer 28

• Binary (yes/no)
• Likert scale (no, some, a lot)
• Continuum (0-100)

Question 29

What are the 7 steps to create a measure?

Answer 29

1. Theory development (conceptualise your measure)
2. Item generation (write out your items)
3. Scoring development (how your answers will be scored)
4. Content validation (evaluate content of items)
5. Reliability assessment (test reliability of measure)
6. Construct validation
7. Criterion validation ( evaluate extent to which measure is useful in explaining important outcomes)

Question 30

How do we carry out theory development? Include questions that you must ask yourself during it.

Answer 30

• What precisely will this test measure? (theoretical model)
• Have a clear and specific definition (review previous research about the topic)
• Measures should match your theoretical constructs (select the behaviours that are representative of the attribute)
• What is the intended target group?
• What will it be used for?

Question 31

Define ‘unidimensionality’.

Answer 31

Items and scale must measure only the attribute and nothing else.

Question 32

What types of question should you use during item generation?

Answer 32

• ‘To what extent do you agree with the following questions’
• ‘Rate how satisfied you are with each of the following’

Question 33

What is important to consider during item generation?

Answer 33

• Ask experts to identify which items belong.
• Eliminate or rewrite any item where fewer than 80% of experts agree.

Question 34

What should we avoid whilst generating items?

Answer 34

• Ambiguity, confusion and vagueness
• Lengthy wording of items
• Double barrelled questions
• Negatively worded items
• Double negatives, e.g ‘Is it not true that I don’t…’
• Hypothetical items, e.g. ‘if you were in X situation, how would you behave’

Question 35

What are the two different types of responses?

Answer 35

• Qualitative
• Quantitative (can be numbered)

Question 36

What are the issues with open-ended questions?

Answer 36

• Large amounts of missing data
• Leave enough room for people to respond

Question 37

What are the issues with nominal measurement?

Answer 37

• Loses information
• Categories must be clear
• Differences between categories must make sense

Question 38

List and define the three types of reliability.

Answer 38

• Internal consistency : whether the items we use consistently measure the same thing (we use coefficient alpha for this)
• Inter-rater agreement : correlation between scores of two or more raters
• Temporal stability : correlation between scores at two times

Question 39

What is the importance of internal consistency?

Answer 39

• Assesses the consistency of items supposed to capture the same construct
• Shows how related the items are to each other

Question 40

What are the characteristics of the coefficient alpha?

Answer 40

• Always a number from 0 to 1
• Percent of the ‘true score’ variance represented by the measure

Question 41

What are the three ways to increase the coefficient alpha?

Answer 41

• Phrase questions more clearly
• Add items
• Drop poor items (those with a value <0.70)

Question 42

Give an example of disease specific measure.

Answer 42

They relate to a specific health condition, e.g. the Asthma Quality of Life Questionnaire (AQLQ) to measure the functional problems that are most troublesome to adults with asthma.

Question 43

What is an advantage of disease specific measures?

Answer 43

Targeted focus of disease-specific instruments can make them clinically relevant.

Question 44

What is a disadvantage of disease specific measures?

Answer 44

It is not generally possible to administer disease-specific instruments to samples who do not have the relevant health problem.

Question 45

What is an advantage of generic measures of health?

Answer 45

They will pick up other effects of treatment.

Question 46

What is a disadvantage of generic measures of health?

Answer 46

They may not be sensitive to small, but important, health effects.

Question 47

What is SF-36?

Answer 47

A 36 item instrument that measures health across dimensions of physical and social functioning, role limitations due to emotional and physical problems, mental health, vitality and pain.