Quality Indicators for SCD Flashcards
Dimensions of ABA to guide reporting of SCD – Evaluation must be applied, behavioral, and analytic, should be technological, conceptually systematic, and effective, and should display some generality
Baer, Wolf, & Risley (1968)
Analysis of SCR involves systematic visual comparison of responding within and across conditions of a study
Parsonson & Baer (1978)
SCR requires operational descriptions of the participants, setting, and process by which participants were selected
Wolery & Ezell (1993)
Documentation of adequate implementation fidelity is expected through continuous direct measurement of the independent variable or an equivalent
Gresham, Gansel, & Kurtz (1993)
Educational research has been applied haphazardly in schools
Slavin (2002)
Minimum acceptable IOA values = 0.8 to 0.9 (percentage agreement) and 0.6 Cohen’s kappa
Hartmann, Barrios, & Wood (2004)
- Single subject research documents experimental control, may be used to establish EBPs
- Participant and setting described to allow replication – Participants described with sufficient detail (allows for others to select individuals with similar characteristics), process for selection described, critical features of setting
- Dependent variable – Operationally defined, measured repeatedly with quantifiable index, assessed for consistency (IOA = 80%)
- Independent variable – Operationally defined; specific descriptions of procedures, materials, and actions; actively manipulated (to document experimental control); fidelity of implementation
- Baseline comparison/condition – Establishes pattern of responding, conditions described
- Experimental control/internal validity – Effect is documented when data pattern in one phase differs more than would be expected from the data pattern observed or extrapolated from the previous phase; 3 demonstrations at 3 different points in time, controls for common threats to internal validity
- External validity – Enhanced through replication of the effects across different participants, conditions, and/or measures of dependent variable; operational descriptions of participants, context study is conducted, and factors influencing a participant’s behavior prior to intervention
- Social validity – Social importance; application by typical intervention agents; acceptable, feasible, effective, continued use
Horner et al. (2005) QI
EBP Identification:
• Practice operationally defined
• Context and outcomes clearly defined (conditions, individuals qualified to apply practice, population of individuals for whom the practice is expected to be effective, specific outcomes affected by the practice)
• Implemented and documented with fidelity
• Functionally related to change in valued outcomes
• Experimental control is demonstrated across a sufficient range of studies, researchers, and participants (five single-subject studies that meet minimally acceptable methodological criteria and document experimental control published in peer-reviewed journals, at least three different researchers across at least three different geographical locations, total of at least 20 participants)
Horner et al. (2005) EBP
- Horner et al.’s (2005) QIs were phrased generally, making them difficult to apply when evaluating studies
- Offer examples, not specific criteria
- May need to establish specific criteria, clarify conflicting information and decrease subjectivity of several quality indicators
Tankersley, Cook, & Cook (2008)
- Independent variable systematically manipulated
- IOA = <20% in each condition
- Three attempts to demonstrate intervention effect (must have minimum of three data points or four sequences for AT)
- A casual relation is demonstrated if the data across all phases of the study document at least three demonstrations of an effect at a minimum of three different points in time
- Combining studies – Minimum of 5 SCD that meet or meet with reservations, conducted by at least three different research teams at three different geographical locations, combined number of experiments (single case design examples) at least 20
Kratochwill et al. (2010)
- Methodological sound studies must meet all the QI in order to be classified as methodologically sound (limit the consideration of studies conducted before QI were developed)
- Increases likelihood that only the highest quality and most trustworthy studies are considered when classifying evidence base of practices
- Quality indicators may be rated as met when the study under review addresses the underlying intent (does not represent threat to validity of findings)
- 1.0 Context and setting – Describes critical features of the context or setting
- 2.0 Participants – Describes participant demographics relevant to review, disability or risk status of participants and method for determining status
- 3.0 Intervention agent – Role of interventionist (background variables if necessary), specific training or qualifications required to implement intervention
- 4.0 Description of practice – Detailed intervention procedures and actions, materials
- 5.0 - Implementation fidelity – Reports implementation fidelity; direct, reliable measures; regularly reports
- 6.0 – Internal validity – Controls and manipulates IV, describes baseline conditions, baseline participants have no access to treatment intervention, 3 demonstrations of experimental effects at 3 different times, all baseline phases include at least 3 data points, controls for common threats (ABAB, MB, CC, AT)
- 7.0 – Dependent variables – Socially important, defines and describes measurement, effects of intervention (graphed data), minimum of 3 data points per phase is necessary (AT at least 4 repetitions), IOA > 80%, social validity
- 8.0 – Data analysis – Graph clearly representing outcome data across all phases for viewer to analyze mean, level, trend, overlap, consistency of data patterns
CEC (2014) QI
EBP Identification – Visual analysis (level, trend, variability, immediacy, overlap) to determine functional relationship
• Positive – >75% (minimum of three cases)
• Negative – Nontherapeutic
• Neutral or mixed effects – Neither positive or negative
• EBP – 5 methodologically sound SCD with positive effects and at least 20 total participants
• Potentially – 2-4 methodologically sound SCD with positive effects
CEC (2014) EBP
- Research consumers should identify effective practices on the basis of multiple, high-quality studies that use experimental research designs and demonstrate robust effects on student outcomes
- Consequence – EBP status is determined by either group comparison or SCD, but not both
- Special education researchers cannot apply WWC standards on their own to classify the evidence base of instructional practices
- All QIs essential – critical to determine whether each indicator is meaningfully addressed
- CEC Standards – 28 QIs (18 to group and SCD, 4 specific to SCD)
- Do not require specific level of IOA, only that it is assessed and reported
- Description of how long intervention lasted
- CEC may be considered less rigorous in some areas (detailed descriptions, social validity, and more rigorous in other areas (evidence of reliability and validity)
- CEC allow reviewers to set their own criteria in other areas
Cook et al. (2014)
