Measurement concepts & methods Flashcards
measurement
process of assigning numbers to objects, events, or situations in accord with some rule
instrumentation
application of rules to develop a measurement device or instrument (scale, questionnaire, etc.)
direct measurement
concrete, observable
indirect measurement
abstract variables/concepts
indicators or attributes are measured, usually with multi-item scale
measurement error
difference between true value and measured value
can be random or systematic
can occur in both direct and indirect measurements
random error
causes individuals’ observed scores to vary in no particular direction around true score
observed score = true score + random error
bc true score is never known, random error can only be estimated
systematic error
consistent across measures - always high or always low
observed score = true score + constant (there is a formula for SE)
reduced by calibrating physiological instruments or selecting quality (ie reliable/valid) measurement methods
reliability
measure of CONSISTENCY
if measurement doesn’t change when concept being measured remains constant in value
ex. height: if you use measuring tape to measure ht, you expect to receive similar results each time
* an instrument that is unreliable cannot be valid
validity
the degree to which an instrument measures what it’s supposed to be measuring
addresses appropriateness, meaningfulness, usefulness of specific inferences made from instrument scores
ex. measuring depression: does tool you chose actually measure characteristics of the concept of depression?
* an invalid instrument can still be reliable
reliability testing
usu expressed as a form of correlation coefficient (0.00/none~1.00/perfect)
0.80/higher is strong value for entire measurement scale/inventory
stability reliability
can be repeated over and over on the same subject and produce same result
assumes that variable being measured is constant over time
test/retest
equivalence reliability
attempt to determine if similar tests/observers give same results
types of tests: alternate form (instruments), inter-rater reliability (observers)
internal consistency
extent to which all parts of the instrument measures same concept
correlation of items within test/scale
provides useful measure of reliability in structured quantitative instruments
tests include Cronbach’s alpha
Cronbach’s alpha
most widely used method for evaluating internal consistency
can be interpreted like other reliability coefficients (ie 0.00~1.00) - higher values reflect higher internal consistency
face validity
instrument looks valid/gives appearance of measuring what it’s supposed to measure
every measure should be inspected for face validity
this alone doesn’t provide convincing evidence of measurement validity
subjective!
content validity
extent to which measurement method includes all major elements RELEVANT to construct being measured (domain)
determined by content experts, scholars, literature
concurrent validity
measure yields scores that are closely related to scores on a criterion measured at the same time
predictive validity
ability of measure to predict scores on a criterion measured in the future
validity from factor analysis
determines which items on a test are closely related to one another
determines dimensions/subcomponents (factors) of a phenomenon
exploratory factor analysis (EFA)
confirmatory factor analysis (CFA) - confirm composition of final instrument
critiquing reliability
where is reliability reported?
find evidence of reliability of instruments used
what type of reliability did they assess?
what statistic did they come up with?
how good would you say it was?
was reliability assessed for instruments in context of the study?
critiquing validity
find any mention of validity of instruments used
what kind of validity?
how assessed?
physiological measurement
biophysical (eg BP) or biochemical (eg lab values)
can be obtained directly (observation) or indirectly (self-report)
accuracy & precision!
accuracy
determining closeness of agreement between measured value and true value
two values are tested for correlation - anything less than 95% is suspect
depends on quality of measurement equipment/device, detail of data collection plan, expertise of data collector
research reports contain details of how devices and equipment are used
precision
degree of consistency or reproducibility of measurements made with physiological instruments or devices
very similar to test-retest reliability
observational measurement
usually visual
must be pilot-tested to generate data on inter-rater reliability
define what specific behaviors/events are to be inspected/observed in study (checklist/counter, category systems, rating scales)
interviews
verbal communication
most common in qualitative/descriptive studies
pilot the questions - they may be ambiguous
make sure all research assistants are similarly trained
questionnaires
printed self-report form
lend themselves to quantitative items
designed to determine facts about persons, facts about events/situations or to measure beliefs, attitudes, opinions, etc.
designed before data collection begins
same questions, same order each time!
development of questionnaires
blueprint/table or specifications
items have two parts: lead-in question (stem), response set (dichotomous, ordinal, ratio)
terms must be defined
value-free questions
group question according to topics
demographics first or last
scales
more precise than questionnaires
self-report
single-item vs. multiple-item
types: rating, Likert, semantic differentials, visual analog
rating scales
lists of categories of a variable
each category represented by a number
assumption of continuous values
common for valuing: “rate your ___”
one of the crudest forms of scaling
eg Wong-Baker pain scale, numeric pain scale
Likert scale
determines opinion or attitude of subject
most commonly used scale in nursing
most commonly addresses agreement, evaluation, or frequency
“never/sometimes/frequent/always”
semantic differentials
measures varying degrees of positive and negative attitudes, beliefs, PoV, opinions
2 opposite adjectives with 7-point scale between them
visual analog scale
measures magnitude, strength, intensity of individual’s sensations/feelings
useful in scaling stimuli (eg pain, anxiety, quality of sleep, etc.)
diaries
individual’s subjective documentation
captures immediate perception
narrative, fill in blanks, selecting best response from list of options, checking a column
measurement using existing databases
adv: no-work data collection (already done), inexpensive, may reveal unexpected relationships b/w variables
disadv: not every variable useful, level of measurement may limit analysis techniques, reliability/validity concerns
evaluating existing instrument for appropriateness/performance
does it match conceptual definition?
is there info on instrument’s reliability and validity in similar research?
is instrument adequately sensitive?
what does instrument cost?
special training required?
is instrument reasonable regarding time required to complete it?
