Measurement concepts & methods

Question 1

measurement

Answer 1

process of assigning numbers to objects, events, or situations in accord with some rule

Question 2

instrumentation

Answer 2

application of rules to develop a measurement device or instrument (scale, questionnaire, etc.)

Question 3

direct measurement

Answer 3

concrete, observable

Question 4

indirect measurement

Answer 4

abstract variables/concepts

indicators or attributes are measured, usually with multi-item scale

Question 5

measurement error

Answer 5

difference between true value and measured value

can be random or systematic

can occur in both direct and indirect measurements

Question 6

random error

Answer 6

causes individuals’ observed scores to vary in no particular direction around true score

observed score = true score + random error

bc true score is never known, random error can only be estimated

Question 7

systematic error

Answer 7

consistent across measures - always high or always low

observed score = true score + constant (there is a formula for SE)

reduced by calibrating physiological instruments or selecting quality (ie reliable/valid) measurement methods

Question 8

reliability

Answer 8

measure of CONSISTENCY

if measurement doesn’t change when concept being measured remains constant in value

ex. height: if you use measuring tape to measure ht, you expect to receive similar results each time
* an instrument that is unreliable cannot be valid

Question 9

validity

Answer 9

the degree to which an instrument measures what it’s supposed to be measuring

addresses appropriateness, meaningfulness, usefulness of specific inferences made from instrument scores

ex. measuring depression: does tool you chose actually measure characteristics of the concept of depression?
* an invalid instrument can still be reliable

Question 10

reliability testing

Answer 10

usu expressed as a form of correlation coefficient (0.00/none~1.00/perfect)

0.80/higher is strong value for entire measurement scale/inventory

Question 11

stability reliability

Answer 11

can be repeated over and over on the same subject and produce same result

assumes that variable being measured is constant over time

test/retest

Question 12

equivalence reliability

Answer 12

attempt to determine if similar tests/observers give same results

types of tests: alternate form (instruments), inter-rater reliability (observers)

Question 13

internal consistency

Answer 13

extent to which all parts of the instrument measures same concept

correlation of items within test/scale

provides useful measure of reliability in structured quantitative instruments

tests include Cronbach’s alpha

Question 14

Cronbach’s alpha

Answer 14

most widely used method for evaluating internal consistency

can be interpreted like other reliability coefficients (ie 0.00~1.00) - higher values reflect higher internal consistency

Question 15

face validity

Answer 15

instrument looks valid/gives appearance of measuring what it’s supposed to measure

every measure should be inspected for face validity

this alone doesn’t provide convincing evidence of measurement validity

subjective!

Question 16

content validity

Answer 16

extent to which measurement method includes all major elements RELEVANT to construct being measured (domain)

determined by content experts, scholars, literature

Question 17

concurrent validity

Answer 17

measure yields scores that are closely related to scores on a criterion measured at the same time

Question 18

predictive validity

Answer 18

ability of measure to predict scores on a criterion measured in the future

Question 19

validity from factor analysis

Answer 19

determines which items on a test are closely related to one another

determines dimensions/subcomponents (factors) of a phenomenon

exploratory factor analysis (EFA)

confirmatory factor analysis (CFA) - confirm composition of final instrument

Question 20

critiquing reliability

Answer 20

where is reliability reported?

find evidence of reliability of instruments used

what type of reliability did they assess?

what statistic did they come up with?

how good would you say it was?

was reliability assessed for instruments in context of the study?

Question 21

critiquing validity

Answer 21

find any mention of validity of instruments used

what kind of validity?

how assessed?

Question 22

physiological measurement

Answer 22

biophysical (eg BP) or biochemical (eg lab values)

can be obtained directly (observation) or indirectly (self-report)

accuracy & precision!

Question 23

accuracy

Answer 23

determining closeness of agreement between measured value and true value

two values are tested for correlation - anything less than 95% is suspect

depends on quality of measurement equipment/device, detail of data collection plan, expertise of data collector

research reports contain details of how devices and equipment are used

Question 24

precision

Answer 24

degree of consistency or reproducibility of measurements made with physiological instruments or devices

very similar to test-retest reliability

Question 25

observational measurement

Answer 25

usually visual

must be pilot-tested to generate data on inter-rater reliability

define what specific behaviors/events are to be inspected/observed in study (checklist/counter, category systems, rating scales)

Question 26

interviews

Answer 26

verbal communication

most common in qualitative/descriptive studies

pilot the questions - they may be ambiguous

make sure all research assistants are similarly trained

Question 27

questionnaires

Answer 27

printed self-report form

lend themselves to quantitative items

designed to determine facts about persons, facts about events/situations or to measure beliefs, attitudes, opinions, etc.

designed before data collection begins

same questions, same order each time!

Question 28

development of questionnaires

Answer 28

blueprint/table or specifications

items have two parts: lead-in question (stem), response set (dichotomous, ordinal, ratio)

terms must be defined

value-free questions

group question according to topics

demographics first or last

Question 29

scales

Answer 29

more precise than questionnaires

self-report

single-item vs. multiple-item

types: rating, Likert, semantic differentials, visual analog

Question 30

rating scales

Answer 30

lists of categories of a variable

each category represented by a number

assumption of continuous values

common for valuing: “rate your ___”

one of the crudest forms of scaling

eg Wong-Baker pain scale, numeric pain scale

Question 31

Likert scale

Answer 31

determines opinion or attitude of subject

most commonly used scale in nursing

most commonly addresses agreement, evaluation, or frequency

“never/sometimes/frequent/always”

Question 32

semantic differentials

Answer 32

measures varying degrees of positive and negative attitudes, beliefs, PoV, opinions

2 opposite adjectives with 7-point scale between them

Question 33

visual analog scale

Answer 33

measures magnitude, strength, intensity of individual’s sensations/feelings

useful in scaling stimuli (eg pain, anxiety, quality of sleep, etc.)

Question 34

diaries

Answer 34

individual’s subjective documentation

captures immediate perception

narrative, fill in blanks, selecting best response from list of options, checking a column

Question 35

measurement using existing databases

Answer 35

adv: no-work data collection (already done), inexpensive, may reveal unexpected relationships b/w variables
disadv: not every variable useful, level of measurement may limit analysis techniques, reliability/validity concerns

Question 36

evaluating existing instrument for appropriateness/performance

Answer 36

does it match conceptual definition?

is there info on instrument’s reliability and validity in similar research?

is instrument adequately sensitive?

what does instrument cost?

special training required?

is instrument reasonable regarding time required to complete it?