7-Advanced topics in measurement Flashcards
common
qCombining targets sometimes eases analysis
qHowever, we need to analyze the Y-axis
Multiple Targets on One Graph
may be Estimated given the Rate provided that the Variability is not too great, or that there are no Outliers.
formula:
• Recording interval/count
Helps to remember there are: 60 minutes in 1 hour and 3600 seconds in 1 hour.
IRT
- Rate = Responses / Time
If rate is 6 per hour, that is same as 6 times per 60 minutes (6/60 minutes).
IRT = Time Units / Responses
Solve for the average IRT by dividing the time by the number of responses (60 minutes/6).
Therefore estimated IRT is 10 minutes; that is there is about 10 minutes (on average) between the end of one response and the beginning of
another response.
- Rate = Responses / Time
If rate is 120 per hour, that is same as 120 times per 3600 seconds (120/360 seconds).
IRT = Time Units / Responses
Solve for the average IRT by dividing the time by the number of responses (3600 seconds/120).
Therefore the estimated IRT is 30 seconds;
that is there is about 30 seconds (on average) between the end of one response and the beginning of another response.
IRT examples
- total count (Calculated by (Smaller/LargerX 100 )
- percent agreement
(Calculated by (Agreements/Agreements +
Disagreement) X 100).
• yields the same value as total count for direct measurement
•used primarily for indirect measurement methods
.
Also by looking at interval-by-interval.
- Number of intervals agreed to/number of intervals agreed + number of intervals disagreed x 100
• can look at either scored intervals or unscored intervals.pppopp
two main methods determining interobserver agreement
Common to have multiple targets on one Graph
Combining targets on one graph sometimes eases Analysis
However, we need to Analyze the Y-axis.
When Selecting targets to Measure we need to assess if the targets typically occur together. Ex. Aggression and screaming.
Multiple targets on one graph
when targets are related ( co-occur)
If they are all acceleration or deceleration targets of similar scales
if all targets are scaled approximately the same (or use dual axes
Multiple targets may be graphed together..
It may be Appropriate to measure more than one Dimension of a behavior.
Commonly, Frequency and Duration are measured together to provide a more complete picture.
Consider Latency and Duration
Perhaps data are collected on one Dimension until target behavior occurs.
If aggression is rare, but tends to occur in “Bursts”, may be interested in rate or IRT for period of
time after an instance of Aggression is recorded.
Alternatively, some dimensions are occasionally included in Definitions
Ex, Duration As a requirement and tantrums; latency in compliance.
Multiple dimensions
Personal experience warning: I have found that it is easier to Present multiple dimensions of the same target on one graph when the data paths are Separated to allow visual analysis.
If not the case, use Multiple graphs.
Multiple dimension presentation
Often multiple Dimensions of a single target are included on a single graph.
Can Collapse dimensions as part of
Definition of target..
-May be beneficial
-loses some precision
Multiple dimensions displayed on a single graph can be beneficial if:
- dimensions are Scaled approximately equally (or use multiple axes)
- data paths are sufficiently Separate to allow visual analysis.
Take Aways
Procedure designed to Decelerate behavior oProtect clients that temporarily limits movement or access to reinforcers.
procedures designed to keep individuals safe (most common examples of target behaviors include aggression, self-injurious behavior, and property destruction)
designed to Decelerate behavior (Typically more severe/resistant problematic behavior
May protect individuals and have other affects as well.
If these are going to be used they must always be tracked
Most often, the target behavior that necessitates the restrictive procedure is tracked along with the procedure
restrictive (also called restrictives) i
A procedure that is used to protect a client should he or she engage in behavior that is dangerous to self or others, has the potential to cause significant property damage, or expose the client to legal consequences
These procedures are not part of the treatment plan
Almost always some form of reporting structure for this procedure, i.e. incident report
Even if not, should collect as much information about incident as possible
These data off and yelled significant information concerning target behavior
Emergency procedures
Always graph restrictive
May or may not graph emergency procedures
When feasible graph restrictive procedure and target together
Some takeaways/emergency procedures
BC BA level clinicians will probably not be taking all of the data for a given client.
Therefore, collect data on staff implementation
Monitoring staff Implementation a.k.a. integrity checks, important for:
• Acquisition: provides information on teaching procedures
• Deceleration: provides information on effectiveness and proper implementation
Staff Implementation
are taught to graph everything
However it is possible to go overboard
Fairly adept at presenting data
If we choose to do so, we could hide information in graphs
Must uphold rigorous standards to avoid missed leading others
Behavior analysts –Misuse of data
Largely automated but do not have to be
instances of responding/data
- Words written in a dissertation. Only the number of words you get down in a month is of interest.
- Deaths in an Industry
- Exercise
Cumulative records
Important in clinical work (tiger)
Could use:
•percent selected per option
• Time spent
• cumulative recording
Choice
(1) Determine what one Trial will be (for example, untied shoe).
(2) Decide how to Report number of trials or number of block trials), for example, when tying shoes could use Trials
but for cleaning the kitchen might use Blocks of trials.
(3) Record Count
If using block it would be number of trials before one can complete all of the steps in the block
- Present data
Steps when conducting trials-to-criterion
Evaluate the efficacy of different teaching strategies
Assessing learner confidence
Limitation: behavior that is difficult to count
Ex. Precision teaching versus discrete trials
Homogeneous ability groups versus random classroom assignments
Trials to criterion: when to use
(Review definition)
Method for classifying responses into discrete categories
Expressed as percentage of responses for each code
Similar to frequency but,
-Most often used when several different behaviors occur
-there’s limited number of opportunities to Observe
Ex: independence with washing hands. Use task analysis form with codes: i= independent. VP = verbal prompt; GP = gestural prompts etc.
Ex. severity codes, duration codes, independence codes
Limitations : not dimensional quantities
Discrete categorization – coding
Targets do not have to be related like handwashing
• Aggression 1: makes contact with another person
• Aggression 2: leaves red mark
•Aggression 3: brakes skin
Discrete categorization – considerations
• Can EASE data collection.
Ex., if you ask a teacher or parent to collect data on academic skills, you could use CATEGORIES such as
Academic work (A): 10 minutes uninterrupted, Academic work (B): 20 minutes uninterrupted, Academic work (C): 30 minutes uninterrupted.
Discrete categorization
A method of quantifying the chances of obtaining a specific sample from a population
Determined by a fraction:
Sample/population
Sample = number of items meeting definition Population = total number of items
t is not always possible, or advisable, to conduct assessments on problematic behavior.
But, if other behavior Reliably occurs before target behavior you can assess that instead.
- example, if someone first engages in grinding teeth or hand-wringing and then aggression.
Probability want to come over there and come over there
Expressed from zero - Cannot happen to 1- always happens
Pick a marble from HR. They are 35 red marbles and 65 black marbles. What is the probability you will get a red marble?
35/100 = .35
Not 35/65 = .54
Probability
• common in ABA (this is how we collapse data).
• calculated by taking the number of Times an event occurred per 100 Opportunities that the event could have occurred. ex.: divide the number of students who have blue eyes in class by the total number of students and multiply this by 100.
In ABA usually used to talk about correct answers
- ex, 88 out of 100 math problems correct would be 88%)
- number of intervals (for example, aggression in 19 out of 20 intervals would be 95%).
Percent
Consider three percentages:
• 25% chance of rain tomorrow
• 90% chance that the surgery will be successful,
•!child was aggressive in 45% of the intervals.
Small denominators must be viewed with extreme caution.
Think of the change a small difference can make, such as 5 out of 10 versus 6 out of 10.
Cooper, Heron, and Heward (2007) do not recommend calculating percentages with less than 30 opportunities.
Beware of misleading data, for example, if on day 1 a client gets 12 out of 20 questions correct (60%) and then day 2 the same client gets 19 out of 30 questions correct (63%). He’s getting better, right?
What Does Percent Mean?
will cap you at 100% so compare this to a Fluency measure
How do you Interpret a change in percentage?
- ex; if Josh got 15% correct yesterday and 30% correct today, how would you interpret that? You might say performance increased by 15 Percentage points, but his performance Improved by 50%?
Percent
If i increases responding from 10% to 12%,
• 12 divided by 10 is 120% so it is a 20% rise.
•But saying this may make people think you mean that Responding went from 10% to 30%.
Or you could say that 10% to 12%, is a 2% rise. But saying this makes people think you increased from 10% to 10.2%
• Use percentage points (2) and percent increase (20%).
• all of this confusion is Avoided by using counts.
When evaluating percent, if
IRT may be estimated given the rate,
provided that the variability is not too
great, or that there are no OUTLIERS
qThe formula is: § Recording Interval / Count qHelps to remember there are: § 60 minutes in 1 hour § 3600 seconds in 1 hour
Estimating IRT’s
Rate = Responses / Time
If rate = 6 per hour
That is same as 6/60 minutes
IRT = Time Units / Responses
qSolve: 60/6
qEstimated IRT = 10 minutes
Example: Estimate the IRT
May be used for interpretation
Describes Properties of samples.
Used with groups
Used with repeated Measures of an individual.
Complements visual analysis
Often used by behavior analysts for level and IOA
Descriptive statistics
Include:
Mean
Median
Mode ( 10, 15, 22, 25, 25, 27, 32, 34). 25, 25 = Mode
Popular descriptive statistics
When compared to median, Mean is a more sensitive measure
May choose to report MEDIAN data if you have significant outliers.
Descriptive statistics
Range
Standard deviation
Effect size
Descriptive statistics – less common
Typically reported if;
- you collapse data for a single individual
- Intervene on a group
If you intervene on groups you may not be able to pull out Individuals
Mean
The 80–20 rule
In business, often acknowledged that 80% of revenue comes from 20% of customers.
A robust theory
The richest 20% have about 80% of wealth.
80% of Complaints come from about 20% of customers
80% of injuries come from about 20% of Hazards.
Pareto Principal
If we consider a group of individuals like clients who engage in aggression, it becomes important.
Have to decide where to invest your time, effort, and money.
80% of your client referrals will come from?
To make the biggest impact, find that 20%.
Ex., School setting that is out of control
Pareto Principle- Why it applies to us
Comes from searching the “archives” for data that have been Recorded in the past
Different from a records review.
-Archival data collection is not common in Practice or research due to availability.
Archival data – non-experimental method
Best uses:
Rule out treatments
Obtain baseline data
Obtain Comparison baselines
Archival studies
Participants are asked to respond to a set of open or close ended questions.
Best to get anonymous information from large numbers of people
No telling if this information is accurate.
Not common in ABA except occasionally to get a starting point.
Example, what do you consider an Acceptable mastery criteria?
Occasionally to assess individual preferences
Survey data
Relationships between variables are observed and Recorded as they occur naturally
No manipulation so there is no experimental control
Why bother?
- You can’t control everything… But may be able to record it.
Can lead to :
- serendipitous feelings. Ex, staffing patterns
- Treatment hypothesis. Example, agitation and inactivity
Observational studies
Refers to the extent to which measures of behavior under the same environmental conditions Diverge from one another
It is not the range of data
The opposite of stability
Variability in data- SPC
Derning;
Put a large number of white bees in a bucket and then put in a small number of red beads and mix them
Pull out a handful, 25, and there will probably be a few red beards.
Number of red beads could range from 0 to 6.
1. variation expected in a process (0 to 6)
- Variation due to unusual circumstances, 6 or greater
- Common Cause Variation
02. Special cause variation
A method of assessing the variability in a process by using statistical methods
Not population statistics
These are process statistics
Use for constructing charts, depending upon the type of data being assessed
Use for applying a set of decision rules to On-going data
General, typically do not implement changes when there is only common cause variation.
Only implement when there is a special cause variation
Use of Sigma values allows for the application of Decision rules
These procedures may aid, NOT replace a visual inspection
Statistical process control, SPC
Keller; there are a number of these that SPC software Will look for:
- Value beyond the three Sigma value
- Nine consecutive values on one side of the average value
- Two out of three outside of two sigma value, on the same side of the average
- Four out of five consecutive points beyond one stigma, on the same side of the average
- Six increasing or decreasing data points
- 14 points alternating up and down
- 15 consecutive points between plus one Sigma and -1 Sigma
- Eight consecutive points be on plus one Sigma and minus one sigma, Both sides of center.
Run tests
is a statistical term otherwise known as a standard deviation. …
A measurement of variability, defined as:
the range of possible outcomes of a given situation.”
Add a set of data and divide by the number of values in the set to find the mean.
Sigma Value
Pfadt; used for:
measuring Baseline Stability
Determining if intervention was successful
Monitoring interventions in real time
Occasionally, baseline can be hard to Interpret, or, Variables Can be missed , or.
Unusual incidences and baseline a point toward a treatment
SPC- Baseline stability
qA method of assessing the variability
in a process by using statistical
methods
qPopulation versus process statistics
Statistical Process Control
Typically adequate in ABA to determine the success of an intervention but…
sometimes Subtle Results required.
Visual inspection
A word on socially significant outcomes.
See video
Sometimes behavior analyst will have individuals on their caseload for long periods of time
May be helpful in alerting us when Changes occur
SPC- Monitor data
