Section 6: Measurement Flashcards
1
Q
3 Dimensional Qualities
A
RTT- Rat-a-Tat-Tat
-
Repeatability
- Count
- Rate (Frequency)
- Celeration
- Temporal Extent - Duration
-
Temporal Locus
- Response Latency (Latency)
- Interresponse Time (IRT)
2
Q
Repeatability (3 Types)
A
(Countability)
CRC
- when behavior can be COUNTED
- instances of a response class occur repeatedly through time
- 3 types
- Count
- Rate (Frequency)
- Celeration
3
Q
Count
A
(one of the 3 types of repeatability)
- FORMULA: add up the behaviors or items
- by itself, count does not give enough information to make decisions regarding a client’s behavior itnervention program
- it is most useful when observation time is constant across observations
- ex. 12 jelly beans in a jar
- ex. 5 instances of punching the therapist
4
Q
Rate/Frequency
A
(one of the 3 types of repeatability)
- FORMULA: add up the behaviors and divide by time
- Ex. Sam ate 30 pickles in two minutes = 15 pickles per minute
- most popular method in ABA
- Rate/frequency is reported as “per second/minute/day/week/month/year”
- the unit of time must be STANDARD within the study
- use rate when you are recording FREE OPERANTS- behaviors that have discrete beginning and ending points
5
Q
Celeration
A
(one of the 3 types of repeatability)
- FORMULA: count per unit of time/time (frequency/time)
- measures of how rates of response change over time
- celeration can mean accelerate or decelerate
- a minimum of 7 measures of rate is recommended to calculate celeration
- Response rate is displayed on the vertical y axis
- Time in days (calendar) is displayed on the x axis
- Celeration Trend Line- a straight line drawn through the graphed data representing the direction and degree of the trend
6
Q
Temporal Extent
A
- when the duration of behavior can be measured
- every instance of behavior occurs during some amount of time
- use duration when
- you wan to measure the amount of time a behavior occurs
- for behaviors that occur for too long a period of time or too short a period of time
- high rate behaviors
- ex. rocking, on task and off task behaviors, humming
- 2 Methods for Calcualting Duration:
- TOTAL DURATION PER SESSION- cumulative amount of time a person engages in the target behavior in the total session (add up all durations)
- DURATION PER OCCURENCE- duration of time that each instance of the behavior occurs
7
Q
Temporal Locus
A
- measuring the time at which behavior occurs
- LOCUS = POINT IN TIME
- examines when an instance of behavior occurs with respect to other events
- 2 types
- Response Latency (Latency)
- Interresponse Time (IRT)
8
Q
Response Latency
A
(Latency)
(one of two types of temporal locus)
- time between onset of a stimulus and initation of a response
- should be used to measure how much time occurs BETWEEN an opportunity to emit a behavior and when the behavior is initiated
9
Q
Interresponse Time
A
(one of two types of temporal locus)
- amount of time that elapses between two consecutive instances of a response class
- IRT should be used when time between responses is important
- often reported by mean or median and range of IRTs per session
- functionally related to rate of response -
- long IRT = slow rate
- short IRT = fast rate
- ex. you can measure IRT between taking bites of food. short IRT for eating fast. long IRT for eating slow.
10
Q
2 Derivative Measures
A
- Percentage
- Trials to Criterion
11
Q
Percentage
A
(one of 2 derivative measures)
- RATIO formed by combining the same dimensional quantities such as count
- expresses the proportional quantitiy of some event in terms of the number of times the event occured per 100 opprotunites that the event could have occurred
- recommended minimum observation intervals or response opportunites = 30
- Advantages
- used when you want to document the percentage of correct responses
- Disadvantages
- does NOT have a dimensional value
- imposes lower and upper limits on the data
- you cannot use it to record proficiency or fluency
- you cannot claim improvement over 100% occured
- ex. a reader reading 100% of the words presented correctly cannot improve in terms of this measure and this number does not illustrate how many words were presented. it could have been 1 or 50 words
12
Q
Trials to Criterion
A
(one of 2 derivative measures)
- a measure of the number of response opprotunites needed to acheive a predetermined level of performance criteria
- depends on the nature of the target behavior and the desired performance level
- can use: count, rate, duration, and latecy measures to determine trials to criterion data
- often used to compare the relative efficiency of two or mroe treatments
- useful for assessing a learner’s competence in acquiring a related class of concepts
- Ex. use for skills such as shoe tying, as each opporutinity can be considered a trial– 11 trials were required to master shoe tying
- Ex. use for discrete trial- it took 20 trials for the learner to learn red vs. not red
13
Q
2 Definitional Measures
A
- Topography
- Magnitude
14
Q
Topography
A
(one of 2 Definitional Measures)
- form or shape of the response
- malleable dimension of behavior - responses of varying form are SHAPED and selected by their consequences
- Topography DOES NOT EQUAL function
15
Q
Magnitude
A
(one of 2 Definitional Measures)
(Force; Intensity; Severity)
- certain responses need to be emitted at specific levels of intensity
- Ex. volume of voice needs to be at a certain magnitude to be defined as yelling
16
Q
Continuous Measurement Procedures
A
- event recording (count, rate/frequency, celeration) and timing (duration, latency, IRT)
- measurement conducted in a manner such that ALL instances of the reponse class of interest are detected during the observation period
- Advantages: useful for behaviors that:
- are free operant
- have a discrete beginning and end
- require minimal displacement of the organism in time/space
- can be emitted at any time
- do not require a lot of time for completion
- Disadvantages: not useful for behaviors that:
- occur at high rates
- are measured via discrete trials
- occur for extended periods of time
- are opportunity based
17
Q
Discontinuous Measurement Procedures
A
- time sampling/interval recording methods
- measurement conducted in a manner such that some instances of the response lass of interest may NOT be detected
- Advantages: useful for behaviors that:
- occur at high rates
- occur for long durations of time
- are measured via discrete trials
- are measured via percentage
- Disadvantages: not useful for behaviors that:
- are free operant
- it is important to obtain every occurrence of that behavior
- require constant attending by observer
18
Q
3 Procedures for Measuring Behavior
A
ETT = Evan Times Tantrums
- Event Recording (continuous)
- Timing (Continuous)
- Time Sampling (Discontinuous)
19
Q
Event Recording
A
(one of the 3 Procedures for Measuring Behavior)
- methods to record the number of times a response occurs
- advantages:
- fairly accurate method
- simple to implement
- great to use with free operant behaviors- each response has a discrete beginning and end
- disadvantages:
- cannot use for responses that occur at very high rates (hand flapping), continuous behaviors (on task behavior, humming) and/or DTT data
20
Q
Timing
A
(one of the 3 Procedures for Measuring Behavior)
- duration
- response latency
- interrresponse time
21
Q
Time Sampling
A
(one of the 3 Procedures for Measuring Behavior)
(Interval Recording; Discontinuous Measurement Systems)
- methods for recording behavior during intervals or at specific moments in time
- gives us an approximation of the actual instances of behavior
- 4 Types: (for CONTINUOUS behavior or high rate bx)
- Whole Interval Recording- UNDERestimates
- Partial Interval Recording- OVERestimates
- Momentary Time Sampling- both or neither
- Planned Activity Check (PLACHECK)
- Advantages:
- good for continuous and or high rate behaviors
- Disadvantages:
- do not use for recroding important but fairly infrequent behaviors
- Ex. baby crawling out of crib at night once a week
22
Q
Continuous Behaviors vs. Continuous Measurement Procedures
A
- continuous behaviors do NOT have a clear beginning and ending
- ex. shouting, humming, sucking thumb, rocking,
- for continuous behaviors, use a DIScontinuous measurement procedure
- continuous measurement procedures- event recording and timing methods
23
Q
Whole Interval Recording
A
UNDERESTIMATES RATE OF BX
(one of 3 types of Time Sampling Procedures)
- record whether the target behavior occured throughout the entire interval
- data is reported as a percentage of the total of intervals in which the target behavior occured the whole time
- Advantages:
- best for measuring a behavior you want to increase (on task behavior) because it would underestimate the occurence of the behavior you want to increas
- Disadvantages:
- not good for when you want to decrease behavior
- you must observe the client for the entire interval
24
Q
Partial Interval Recording
A
OVERESTIMATES RATE OF BX
(one of 3 types of Time Sampling Procedures)
- record whether the target behavior occured at any time during the interval
- used to represent the proprotion (in percent form) of the entire observation period that the behavior occured
- advantages
- easy to meausure multiple behaviors concurrently
- best for measuring a behavior you want to decrease because it would overestimate the rate of problem behavior
- disadvantages
- not good for increasing target bxs
- must observe the target behavior throughout the entire interval
25
Momentary Time Sampling
**OVER or UNDER estimates or NEITHER**
(one of 3 types of Time Sampling Procedures)
* record whether the behavior occured only at the end of the interval when the timer goes off
* report as a percentage
* advantages
* you do not have to continuously measure throughout the entire interval
* disadvantages
* much of the behavior of interest is missed or unaccounted for
* to avoid this problem, **keep the intervals short**
26
Planned Activity Check
*(PLACHECK)*
(one of 3 types of Time Sampling Procedures)
* a variation of MTS for groups
* ex. at the end of an interval the teacher records if the entire group of students is engaged in the target behavior
27
Permanent Product
*(Outcome Recording)*
* measuring behavior after it has occurred by measuring the effects the behavior produced on the entironment
* a change in the environment produced by a behavior that lasts long enough for measurement to take place
* **ex post facto measurement**
* natural or contrived outcomes
* event recording, timing, and time sampling can be applied to measurement by permanent product
* **CONTRIVED PERMANENT PRODUCTS-** behaviors that do NOT have a direct effect on the enviornment can be measured by permanent product through videotape, photographs, audiotape, etc.
* Advantages
* do not need to observe the client
* makes inconventiently times behaviors easier to observe (e.g. sleeping patterns, waking up at night)
* measurement may be more accurate, complete, and continuous
* facilitate data collection for IOA and treatment integrity
* enables data collection for behaviors with mulitple response classes
* reduces any potential reactivity
* disadvantages
* responses accountable for producing a particular outcome may vary
* ex. student may get 100% correct on an assignment, but we don't know the behaviors that comprised getting the correct answers
28
When should I use permanent product?
3 questions to ask yourself:
* are you interested in only the outcome of the bheavior? (not the mediating behaviors that are responsible for the outcome)
* is real time measurement needed?
* can the target behavior even be measured by permanent product:
* each instance of the target bx must produce the same permanent product
* permanent product can only be produced by the target behavior and only by the client (not another behavior or another person)
* obtaining a contrived permanent product will not unduly affect the behavior (presence of recording equipment may influence bx)
29
How to select the right measurement system
* ETHICS WARNING- you should select a way to record data that provides the most ethical and valid depiction of the target behavior
* evaluate the dimensions of the behavior
30
Scheduling Observation and Recording Periods
* ETHICS WARNING- because you should always be mindful of the validity (the method measures what it is supposed to measure) of your data, you have to be careful in planning your observation periods so that you actually capture what the target behavior truly looks like
* Elements to consider AND CONTROL FOR:
* time of day
* day of the week
* weather conditions
* task assighments
* physical or social environment
* MOs
* things to consider for the frequency of recording periods:
* how frequently should you record data and under what conditions
* if you want to increase or decrease the rate of behavior, use defined objectives across standard time periods
* if you want to shape new behaviors OR teach discrimination, use moment to moment/trial by trial data recording
* things to consider for the duration of recording periods:
* you should continue to collect data until you see a stable trend
31
3 Indicators of Trustworthy Measurement
_VAR- Very Awesome Results_
1. Validity
2. Accuracy
3. Reliability
32
Validity
VALID = RELEVANT AND REPRESENTATIVE AND LEGIT
VALID ID REQUIRED = legit representation of a person
* 3 elements
* directly measuring **socially signficant** target behavior
* measuring the **dimension** of the target behavior **relevant to the question** or concern about the behavior
* ensuring that the data are r**epresenative of the behavior's occurrence under the conditions** during times that are more relevant to the concern about the behavior
* measurement has validity, when it yields data that are **directly relevant** to the phenomenon measured and to the reasons for measuring it
* basic question- **was a relevant dimension of the behavior that is the focus of the investigation measured directly and legitimately?**
33
Threats to Validity
Validity is threatened by:
* indirect measurement:
* filtered information
* a proxy or stand in for the actual behavior of interest
* no direct access to the behavior of interest
* inferences about private events
* indirect measurement (direct measures will always have more validity than indirect measures
* measuring the wrong dimension of the target behavior
* measurement artifacts
* 3 causes
* discontinuous measurement
* poorly scheduled measurement periods
* insensitive and/or limiting measurement scales
34
3 Causes of Measurement Artifacts
_DIP- a dip in validity_
(affecting Validity)
1. **discontinuous measurement**- some instances of the response class are NOT detected (all time sampling methods are discontinious and cause artifacts)
2. **insensitive and/or limiting measurement scales**- a scale that imposes an artificial floor or ceiling on performance (ex. giving someone a 100 word passage to test their oral fluency--the best they can do is 100 words per minute)
3. **poorly scheduled measurement periods**- recording data on the behavior at a time that does not properly depict the behavior
35
Accuracy
(one of the 3 indicators of trustworthy measurement)
TRUE VALUE
* the extent to which the **observed value matches the true value of an event**
* for something to be considered a true value, it requires special or extraordinary precautions that ensure that all possible sources of error have been removed
* if measurement is not valid, accuracy is moot
* when true values cannot be established, we use IOA and other reliability measures
36
Reliability
(one of the 3 indicators of trustworthy measurement)
* the extent to which a measurement procedure yields the same value when brought into repeated contact with the same state of nature
* same results repeatedly
* poor reliability reveals problems with accuracy and validity
* consistent measurement
* the closer the values obtained by repeated measurement the more reliable they are
* yet even if the measures are found to be reliable, that doesn't necessarily mean they will also be valid
* Ex. scale at home repeatedly says 130 but the scale at the doctor's office says 140. the scale at home was reliable but it was not accurate
37
Threats to Measurement, Accuracy, and Reliability
* human error
* poorly designed measurement systems
* difficult and cumbersome
* too many participants to observe, behaviors to record
* duration of observation period or observation intervals too long or too short
* inadequate observer training
* ongoing training must be provided to minimize **observer drift**
* expectations about what the data should look like
38
Observer Drift
* when observers unknowingly alter the way they measure a behavior
* produces measurement error
* **occurs when observers have a shift in how they interpret the definitions of the target behavior**
* target behavior definitions DRIFT or change over time
* to minimize: retrain observers
39
Measurement Bias
* non random measurement error likely to be in one direction
* **data that overestimates or underestimates the true value of an event**
* **expectations** that a target behavior will occur under certain conditions or change with a certain treatment can influence what is recorded and threaten the accuracy of the measurement
* how to minimize:
* use **naive observers** who don't know the purpose of the study
* **reduce observer reactivity**- observers can be influenced by anticipating how another observer will record data or just the knowledge that others are evaluating their data
* monitor and record data unobtrusively
* use videotaping for a second observer to get IOA
40
Interobserver Agreement
* **refers to the degree to which two or more indepedent observers report the same values after measuring the same events**
* reporting IOA increases the **believability** that the data is trustworthy and deserving of interpretation
* IOA must be obtained during every phase of a study and should be distributed across days of the week, times of day, settings, and observers
* must be obtained for a **minimum of 20% of sessions**
* IOA is reported in 3 ways
* Narrative description
* Table
* Graphic Display
* Acceptable IOA score is **80% or greater**
41
Benefits and Uses of IOA
1. determines the **competence** of new observers
2. detects **observer drift**
3. increases or decreases confidence that the **definition of the target behavior was clear and the measurement code was not too difficult**
4. gives confidence that variability in data is not a function of which observers were on duty, therefore implying that **changes in data actually reflect changes in behavior**
42
3 Requirements for Obtaining Valid IOA
1. observers must use the same measurement system
2. observers must measure the same event
3. observers must be independent
43
Overview of Methods for Calculating IOA
4 Event Recording IOAs- _METT_
* Total Count IOA
* Mean Count per Interval IOA
* Exact Count per Interval IOA
* Trial by Trial IOA
2 Timing/Duration IOAs
* Total Duration IOA
* Mean Duration (or IRT) per Occurrence IOA
3 Time Sampling/Interval Recording IOAs
* Interval by Interval IOA
* Scored Interval IOA
* Unscored Interval IOA
44
Total Count IOA
(one of the 4 Event Recording IOAs)
* simplest method for event recording
* calculated by dividing the smaller of the counts by the larger count and multiplying by 100
* (smaller #/larger #) x 100
* OVERESTIMATES actual agreement
45
Mean Count Per Interval IOA
* divide the observation period into a series of intervals
* record the number of occurrences of the behavior within each interval
* calculate the agreenment between the count of the 2 observers within each interval
* use the agreements per interval as the basis for calculating the IOA for the total observation period
* (interval 1 IOA + Interval 2 IOA + Interval N IOA) / Total number of intervals
46
Exact Count Per Interval IOA
(one of the 4 Event Recording IOAs)
* divide the observation period into intervals and calculate IOA per interval
* the percentage of intervals in which observers recorded the same count
* most restrictive IOA method
* (# of intervals of 100% IOA agreement / total # of intervals) x 100
47
Trial by Trial IOA
(one of the 4 Event Recording IOAs)
* the agreement between 2 observers who measured the occurrence or nonoccurrence of discrete trial behaviors for which the count for each trial/response opportunity can only be 0 or 1
* calculated by comparing the observers' total counts or by comparing their counts on a trial by trial basis
* (# of trials of agreement / total # of trials) x 100
48
Total Duration IOA
(one of the 2 timing/duration IOAs)
* calculated by dividing the shorter of the 2 durations by the longer duration
* (shorter duration / longer duration) x 100
49
Mean Duration (or IRT) Per Occurrence IOA
(one of the 2 timing/duration IOAs)
* this formula can also be used to compute the mean latency per response IOA or the mean IRT per response IOA
* (Duration IOA Bx1 + Duration IOA Bx2 + Duration IOA BxN) / N behaviors with duration IOA
50
Interval by Interval IOA
(one of the 3 time sampling/interval recording IOAs)
*(Point by Point IOA; Point by Point Agreement Ratio)*
* (# of intervals both recorders are in agreement / total # of intervals) x 100
* likely to OVERESTIMATE actual agreement for behaviors that occur at very high or very low rates
* subject to random or accidental agreement between observers
51
Scored Interval IOA
(one of the 3 time sampling/interval recording IOAs)
* (# of intervals both recorders marked an occurrence / # of intervals AT LEAST ONE recorder marked an occurrence) x 100
* minimizes the effects of chance agreements for behaviors that occur at very high or very low rates
* recommended for behaviors that occur at frequencies of approximately 30% of intervals or fewer to avoid misleading IOA
52
Unscored Interval IOA
(one of the 3 time sampling/interval recording IOAs)
* (# of intervals both recorders marked NON-occurrence / # of intervals AT LEAST ONE recorder marked NON-occurrence) x 100
