Measurements of Behavior Flashcards
Operationalism
belief that scientific concepts must be defined in terms of observable operations
Operational Definition
a precise description of a procedure used to empirically test a theoretical concept
Scientific Method
- make observations
- develop hypothesis
- test hypothesis
- gather data
- evaluate data
- evaluate results
- replicate
Variables
aspect of a testing condition that changes with different conditions
Types of Variables
- independent variables (IV)
- dependent variable (DV)
- confound variable (anything happening at the same time as the IV; undesired)
Writing Criteria for Operational Target Behavior Definitions
- accurate
- complete
- concise
- inclusions
- exclusions
Characteristics of Good Definitions
- objective : refers only to the observable
- clear : readable and unambiguous
- complete : delineate boundaries of definition
Purpose of Good Definitions
- precise and concise description
- reliable observation
- accurate recording
- agreement and replication
Testing a Definition
- can you count the number of occurrences? (should be yes)
- will a stranger know what to look for based on definition alone? (should be yes)
- can you break the target behavior down to smaller, more specific components? (should be no)
Behavioral Measurement
- frequency / rate
- duration
- latency
- locus
- topography
- force
Taking data
- event recording
- timing
- time sampling
Event Recording
procedures for detecting and recording the number of times a behavior is observed
Time Sampling
variety of methods for observing and recording behavior during intervals or at specific moments in time
observation is divided into intervals, presence or absence of behavior recorded for each interval
Partial Interval Recording
record whether the behavior happened at any time during the interval.
tends to underestimate high-frequency behavior and overestimate duration
Whole Interval Recording
at the end of each interval it is recorded if the behavior happened during the whole interval.
the longer the interval the more whole interval will underestimate the occurrence of the behavior
Momentary Time Sampling
recorder notes whether the behavior happens at the moment each interval ends.
not recommended for low frequency, short duration behaviors
PLACHECK (Planned activity check)
momentary time sampling for group engagement
Measuring Behavior by Permanent Product
measuring behavior after it has occurred by measuring its effects on the environment
all previous procedures can be applied to permanent product measurement
products can be a natural or contrived
Advantages of Permanent Product Recording
- practitioner free to do other tasks
- measurement of complex behavior
- easier data collection
- possible measurement of inaccessible behavior
- more accurate, complete, continuous
Inter-Observer Agreement (IOA)
- used in direct observation/recording of behavior
- determines whether the target behavior is being recorded consistently
- agreement does NOT equal accuracy
- aka: inter-observer reliability or inter-rater reliability*
Inter-Observer Agreement
IOA Procedure -
two people independently observe and record the same target behavior (DV) of the same individual during the same observation period
recordings of the two observers are compared
% of agreement between observers is calculated
Types of IOA Data Collection
- continuous measurement - frequency + duration
- discontinuous measurement - interval recording
IOA Calculations
frequency recording - divide smaller frequency by the larger (ex. 9/10 = 90% agreement)
duration recording- divide smaller duration by the larger (ex. 48/50 = 96% agreement)
interval recording- session is broken up into short intervals (ex. agreement/(agreement + disagreement) = 4/(4+1) = .80 = 80%)
IOA Target %
80% = minimal acceptable level
90% or higher is preferred
Indicators of Trustworthy Measurement
- validity
- accuracy
- reliability
Validity
- directly measures a socially significant behavior
- measures a dimension of the behavior relevant to the question
- ensures the data are representative
Accuracy
observed values match the true values of an event
Reliability
measurements yields the same values across repeated measurement of the same event
Threats to Validity
indirect measurement
- measuring a behavior other than the behavior of interest
- requires inferences be made about the relationship between those behaviors
- must provide evidence that the behavior measured is directly related to behavior of interest
Measurement Artifacts
misleading data that result from the way behavior is measured:
- discontinuous measurement
- poorly scheduled observations
- insensitive or limiting measurement scales
Threats to Measurement Accuracy and Reliability
- human error
- poorly designed measurement system
1. cumbersome
2. difficult to use
3. complex - inadequate observer training
1. explicit and systematic
2. careful selection
3. train to competency standard
4. on-going training to minimize observer drift - unintended influences on observers
1. observer expectations of what the data should look like
2. observer reactivity when she/he is aware that others are evaluating the data
3. measurement bias
4. feedback to observers about how their data relates to the goals of intervention
Assessing the Accuracy of Measurement
- determines if data are good enough to make decisions
- discovery and correction of measurement errors
- reveal consistent patterns of measurement error
- assure consumer that data are accurate
Assessing the Quality of Measurement
indicators of the quality of data include:
- IOA
- accuracy
- reliability
can report multiple indices to assess data quality
