section 6 - measurement Flashcards
3 dimensional quantities
- repeatability: count, rate/freq, celeration (e.g. # per min per week)
- temporal extent: duration
- temporal locus: response latency, inter-response time
3 dimensional quantities:
- repeatability: countability
- when B can be counted
- INSTANCES of a RESPONSE CLASS (i.e. share same function) occur repeatedly through time
3 types of repeatability measures
a. count:
- most useful when observation time is CONSTANT.
- NOT enough info to make decisions abt interventions
b. rate/frequency: count/time
- most popular data methods
- useful when recording FREE OPERANT: Bx have discrete beginning & ending points
- NOT useful when recording Bx that occur ONLY within limited/restricted conditions (e.g. discrete trial data, trials measured by opportunities)
- NOT use when measuring CONTINUOUS Bx that occur for extended period of time
- per sec, min, day, week, month, year
- report the UNIT OF TIME
- unit of time must be standard within the study, so can compare 2 rates
c. celeration
- count per unit of time/time (i.e. frequency/time)
- measures how rates of response change over time
- acceleration: rates of response accelerate when responding is FASTER over time
- deceleration: rates of response decelerate when responding is SLOWER over time
- at least 7 measures of rate
- response RATE displays on the VERTICAL axis
- time in days displays on the HORIZONTAL axis
- CELERATION TREND LINE: a STRAIGHT line drawn through the graphed data representing the DIRECTION & DEGREE of the trend
3 dimensional quantities:
- temporal extent: duration
- when the duration of a B can be measured
- every instance of B occurs during some amount of time
- use duration when:
- wanna measure the AMOUNT OF TIME of a B
- Bx that occur for too long/short a period of time
- HIGH RATE Bx
- e.g. rocking, on-task, humming
- total duration per SESSION: the cumulative amount of time a person engages in the target B in the TOTAL SESSION
- duration per OCCURRENCE: duration of time that EACH instance of the B
3 dimensional quantities:
- temporal locus:
- measures the TIME at which B occurs: when an instance of B occurs with respect to other events
- locus: POINT in time
- response latency: time between onset of a stimulus & initiation of a response
- use when wanna measure how much time occurs between an OPPORTUNITY to emit a B & when the B is INITIATED
- inter-response time (IRT): amount of time that elapses between 2 CONSECUTIVE INSTANCES of a response class
- use IRT when time between responses is important
- reported by MEAN, MEDIAN, RANGE of IRT per SESSION
- functionally related to rate of response
2 derivative measures
- percentage
- RATIO: combining the same dimensional quantities =, such as count
- express the PROPORTIONAL quantity of some event in terms of the # of times the event occurred per 100 OPPORTUNITIES that the event could have occurred
- at least 30 observation intervals/response opportunities
- advantage: when document the percentage of correct response
- disadvantage:- NO dimensional value
- has lower & upper limits on the data
- can NOT record PROFICIENCY / FLUENCY
* *NOT correct to claim improvement over 100% occurred**
- trials to criterion
- measure the # of response OPPORTUNITIES needed to achieve a PREDETERMINED level of performance criteria
- a trial depends on the nature of target B & the desired performance level
- can use: count, rate, duration, latency to determine trials to criterion data
- use to COMPARE RELATIVE EFFICIENCY of 2 or more treatments
- use for assessing a learner’s INCREASING COMPETENCE in acquiring a related class of concepts
- use for skills such as shoe tying:- each opportunity to tie shoe can be considered a trial –> trials to criterion data reports the # of TRIALS required for the learner to tie shoe correctly
2 definitional measures
- topography
- FORM/SHAPE of the response
- use when FORM of the B is critical
- e.g. basketball, dancing, etc.
- malleable 有延展性的 dimension of B= responses of VARYING form are SHAPED & SELECTED by their consequences
- topography can be different but with SAME FUNCTION - magnitude
- force/intensity/severity of B
- certain responses need to be emitted at SPECIFIC LEVELS OF INTENSITY
e. g. volume of voice
3 procedures for measuring B
- event recording (continuous measurement procedure)
- timing (continuous)
- time sampling (DIScontinuous)
- event recording (continuous measurement procedure)
- record the # of TIMES a response occurs
- device: pencil & paper, wrist counters, hand tally, digital counters, masking tape, pennies, buttons, calculators, etc.
advantage
- fairly accurate
- simple to implement (while engaging in other activities)
- good to use with FREE OPERANT Bx (each response has a discrete beginning & end)
disadvantage
- NOT use when responses occur at very HIGH rate (e.g. hand flapping), CONTINUOUS Bx (e.g. on task B), discrete trial training (DTT) data
- timing (continuous)
- duration
- response latency
- interresponse time
- device: stopwatch
- time sampling (DIScontinuous)
interval recording
discontinuous measurement system
- recording B during intervals / at specific moments in time
- gives APPROXIMATION of actual instances of B
- how*
1. divide the observation period into EQUAL intervals
2. record the presence/absence of B within or at the end of each interval - advantage: good to record HIGH rate/CONTINUOUS Bx
- disadvantage: NOT use when wanna record important but infrequent Bx.
e. g. crawl out of baby crib once per week
continuous Bx vs. continuous measurement procedures
continuous Bx
- NOT have a clear beginning & ending
- NOT discrete
- e.g. shouting, rocking, sucking
- should use DIScountinuous measurement procedure. e.g. time sampling
continuous measurement procedures
- ALL INSTANCES of the response class of interest are detected during the observation period
- e.g. event recording, timing
3 forms of time sampling: measure continuous B/high rate B
- whole interval recording
- partial interval recording
- momentary time sampling
- whole interval recording
- how*
1. divide the observation period into EQUAL intervals
2. at the END of each interval, record whether the B occurred THROUGHOUT the interval (if a B occurred but not for the entire interval, it is recorded as absence)
3. report data in PERCENTAGE: percentage of total intervals that the targeted B occurred
advantage
- best to measure B you wanna INCREASE
disadvantage
- NOT good when wanna DECREASE B: coz you are required to observe the B occurred throughout the entire interval
- underestimated overall duration of the B
- partial interval recording
- how*
1. divide the observation period into EQUAL intervals
2. at the END of each interval, record whether the B occurred AT ANY TIME during the interval
3. report data in PERCENTAGE: percentage of total intervals that the targeted B occurred - represent the proportion of the entire observation period that the B occurred
advantage:
- easy to measure MULTIPLE Bx concurrently
- best to measure B wanna decrease.
- e.g. rocking
disadvantage:
- NOT good to use when wanna increase Bx
- must observe the B occurred throughout the entire interval
- overestimate overall duration of the B
- underestimate rate of the B
- momentary time sampling
- how*
1. divide the observation period into EQUAL intervals
2. at the END of each interval, record whether the B occurred at the END of the time interval ONLY
3. report data in PERCENTAGE: percentage of total intervals that the targeted B occurred
advantage:
- do NOT have to continuously measure throughout the entire interval
disadvantage:
- underestimated rate/occurrence of the B: much B is missed/unaccounted
- to avoid missing B: should keep intervals short & observe frequently
planned activity check
placheck
- for GROUPS
- variation of momentary time sampling
e. g. a teacher observes a GROUP of students at the END of each interval, records the TOTAL # OF STUDENTS engaged in the targeted activity
permanent product : outcome recording
- measure B after it has occurred by measuring the EFFECTS the B produced on the environment
- B produces a change in the environment that lasts LONG enough for measurement
- EX POST FACTO measurement: measure after B has occurred
- natural/contrived outcomes
- contrived permanent outcomes: Bx do NOT have a DIRECT effect on the environment can be measured by permanent product through VIDEOTAPE, PHOTOGRAPHS, AUDIOTAPES
advantage:
- practitioner is free to do other tasks
- easier to observe inconveniently timed Bx. e.g. sleeping patterns
- measurement may be more accurate, complete, continuous
- facilitate data collection for IOA & treatment integrity
- enable data collection for Bx with MULTIPLE response classes
- reduce potential REACTIVITY (temporarily & is in response to an OBSERVER)
disadvantage:
- responses accountable for producing a particular outcome may vary
e. g. a student earns ‘A’ for an assignment: cannot tell how the student prepare & carry out the assignment
when to use:
- only interest in the outcome of the Bx? or mediating Bx?
- is real time measurement needed?
- can it be measured by permanent products?
rules:
- each instance of the B must produce the SAME PERMANENT outcome
- permanent outcome can ONLY be produced by the target B, can NOT be produced by other Bx or other ppl
- contrived permanent outcome will not unduly 过度的 affect the B
- recording equipment for contrived permanent product may cause a person act differently
- obtrusive measurement can affect the environment, then affect the B
select the right measurement system to obtain representative data given the dimensions of the behavior
- *ethics**
- should select a way to record data that provides the most ethical & valid depiction of the target B
- does the B result in tangible product: counting/permanent product are good to use
e. g. # of toy cars assembled - is the B transitory (i.e. feeling): event recording / time sampling may be better
scheduling observation & recording period
- *ethics**: validity (the method measures what it is supposed to measure)
- elements to consider: time of day, days of the week, weather conditions, task assignments, physical/social surroundings, MOs…etc. –> make sure that these elements are EVENLY distributed throughout baseline, intervention, FU –> if not, cannot confidently interpret data
- things to consider with FREQ of recording:
- how freq should you record data & under what conditions
- if wanna increase/decrease rate of B, use your VALIDLY-DEFINED objectives across standard time periods
- if wanna shape NEW Bx or teach clients to differentiate 1 stimulus from another, use moment-to-moment/trial-by-trial data recording
- things to consider with DURATION of recording phases:
- can continue to assess data until the data become stable according to your pre-specified definition of stability
data
primary material to guide & evaluate behavioral work
3 indicators of trustworthy measurement
- validity
- accurary
- reliability
- relative concepts
- each can range from high to low
- validity
3 elements:
a. DIRECTLY measuring socially sig. target B
b. measuring DIMENSION of the target B RELEVANT to the question/concern abt the B
c. ensure the data are representative of the B’s occurrence under conditions during TIMES that are most relevant to the concern abt the B
- measurement has VAILIDITY when it yields data that directly relevant to the phenomenon measured & to the reasons for measuring it
threats to measurement validity
validity is threatened by:
a. INdirect measurement
- secondhand / filtered info
- researcher measure a PROXY or stand-in for the actual B of interest
- useful when there’s NO direct access to the B of interest
- sometimes used to make inference abt PRIVATE EVENTS
- research is responsible to provide evidence that the event measured directly reflects sth abt the B for which the researcher wishes to draw conclusions
- DIRECT MEASURE of target Bx will always have more validity than INdirect measures
b. measure the WRONG dimension of the target B
- more threaten to validity than indirect measurement
- measurement artifacts
3 causes of measurement artifacts
- discontinuous measurement: measurement in which some instances of the response class are NOT detected
e. g. all time sampling methods are discontinuous–> cause artifacts - poorly scheduled measurement periods: recording data on the B at a time that does NOT properly depict the B
e. g. a B happens at 12pm, but data are recorded at 3pm - insensitive &/or limiting measurement scales: a scale that imposes an ARTIFICIAL floor/ceiling on performance
e. g. give a person 100-word passage to read in 1 min to test their oral reading fluency –> max. performance is 100 words per min
- accuracy
- the extent to which the observed value / quantitative label matches the TRUE value of an event
- for sth to be considered a true value requires special/extraordinary precautions that ensure all possible sources of ERROR have been REMOVED
- if measurement is not valid, accuracy is uncertain
how to establish true values for accuracy measures
- true value must be different than the measurement procedures used to obtain the observed values
- when true value cannot be established, researchers must rely on RELIABILITY assessments & measure of IOA to evaluate the quality of their data
- reliability
- the extent to which a measurement procedure yields the SAME value when brought into REPEATED contact with the SAME state of nature
- SAME RESULT REPEATEDLY
- poor reliability reveals problems with accuracy & validity
- CONSISTENT measurement
- the closer the value obtained by repeated measurement, the more reliable they are
- reliability doesn’t necessarily mean validity / accurary
threats to accurary & reliability
- human error
a. poorly designed measurement systems
b. inadequate observer training
c. expectations abt what the data should look like
a. poorly designed measurement systems
- difficult & cumbersome measurement systems: # of participant to observe, # of Bx to record, duration of observation period/intervals
- simplified systems minimize errors
b. inadequate observer training
observers need EXPLICIT & SYSTEMATIC training
- select observers carefully
- train observers to a standard of competency
- provide ongoing training to minimize OBSERVER DRIFT
observer drift
- when observer UNKNOWINGLY alter the way they measure a B
- UNINTENDED changes in the way data are collected may produce measurement error
- occurs when observers have a shift in how they INTERPRET the definitions of the target B
- the target B definitions DRIFT over time
- how to minimize: RETRAIN observers & provide detailed feedback on the accuracy & reliability of measurement
measurement bias
e.g. c. expectations abt what the data should look like
- NONRANDOM measurement error
- errors in measurement likely to be in 1 direction: over/underestimate the true value
- EXPECTATION that a target B will occur under certain conditions / change with certain treatment can influence what is recorded & threaten the accuracy of measurement
- NAIVE OBSERVERS: min measurement bias
- trained observer who is unaware of the study’s purpose &/ experimental conditions in effect
- OBSERVER REACTIVITY: errors result from an observer’s knowledge that others are evaluating the data he/she reports.
- an observer can be influenced by how he/she anticipates another observer will record data
- how to min: monitor & record data unobtrusively
e. g. use videotaping for second observer to observe the same session
interobserver agreement
- the degree to which 2 or more independent observer report the SAME value after measuring the same event
- report IOA increase BELIEVABILITY
benefits & use of IOA
- determine the competence of new observers
- detect observer drift (unintended)
- increase/decrease confidence that the definition of the target B was clear & measurement code was not too difficult
- given confidence that variability in data is not a function of which observer were on duty –> imply that changes in data actually reflect changes in B
3 requisites for obtaining valid data
a. observers must use the same measurement system
b. observers must measure the same event
c. observers must be independent
4 event recording IOAs
- total count IOA = smaller #/larger # x 100%
- simplest method for event recording
- percentage of agreement between the total # of responses recorded by 2 observers
- OVERESTIMATED the extent of actual agreement
- in DTT, use trial-by-trial IOA
- mean count-per-interval IOA = (int1 IOA + int2 IOA + intN IOA)/total # of intervals
- exact count-per-interval IOA = #of intervals of 100% IOA/total # of intervals
- the most STRICT event recording IOA method - trial-by-trial IOA = # of trials of agreement / total # of trials
- for DTT: measure the occurrence / non-occurrence of discrete trial Bx –> the data can only be 0 or 1
2 time / duration IOA
- total duration IOA = shorter (total) duration/longer (total) duration x 100%
- mean duration per-occurrence IOA = (duration IOA B1 + duration IOA B2 + duration IOA Bn)/total # of Bx with duration IOA
- same formula for mean latency-per-response IOA, mean-IRT-per-response IOA
- more conservative & more meaningful assessment of IOA
3 time sampling / interval recording IOA
- interval-by-interval IOA/poin-to-point IOA = # of intervals both recorders agreed / total # of intervals x 100%
- overestimate the actual agreement measuring Bx that occur at very high/low rates
- subject to random/accidental agreement between observers - scored interval IOA = # of intervals BOTH recorders record occurrence / # of intervals AT LEAST 1 recorder recorded occurrence x 100%
- only uses intervals either or both observers scored an OCCURRENCE of the B
- min. the effects of chance agreement for interval data on Bx that occur at very high/low rates
- recommend for Bx that occur at freq of approximately 30% of intervals or fewer to avoid overinflated & possible misleading IOA measures - unscored interval IOA = # of intervals both recorders recorded NON-occurrence /# of intervals at least 1 recorder recorded NON-occurrence x 100%
- only uses intervals either or both observers recorded NON-occurrence of B
- min. the effects of chance agreement for interval data on Bx that occur at very high/low rates
how often & when should IOA be obtained
- during each phase of a study & distributed across days of the week, times of day, settings, observers
- should be obtained for a min. of 20% sessions
how should IOA be recorded
- narrarive description: mean & range of IOA
- table
- graphic display
what are acceptable IOA scores`
no less than 80%
