Module 3 - Standardised Assessments Flashcards
Which stage in the clinical reasoning cycle will you use standardised assessment?
Process information stage
Standardised test:
Features (5)
Formal tests
Tested on normative sample
Correlational research (compare research to other things)
Valid & reliable
Strict administration & scoring procedures
Standardised tests:
2 types
Norm-referenced tests
Criterion referenced test
Standardised tests:
3 positives
Funding
Provides a baseline to assess progress
Psychometrically reliable & valid = consistency
Standardised tests:
4 negatives
- Valid for only population tested
- Expensive (takes time to administer)
- Anxiety provoking for some clients (because they are so strict)
- Reliability & validity may be vulnerable to operational measurement error
Non-standardised test:
2 features
Flexible to individual & contextual need
Naturalistic, functional
Non-standardised test:
4 types
Observations, language sampling
Authentic Assessments (Communication samples)
Dynamic Assessment
Some criterion-referenced tests
Non-standardised test:
3 Positives
Provides information of client in context
Descriptive approach = individualised
Can use measures to compare behaviours to known norms
Non-standardised test:
2 Negatives
Not sufficient for funding without standardised testing
Normative measures depend on clinician analysis & knowledge
What is best? Standardised or non-standardised tests?
Use a combination of both, backed by clinical reasoning
How to choose between standardised & non-standardised assessments:
6 questions to ask
Who are they for? (who are the tests designed for?)
Which aspects of language are assessed?
How are aspects assessed?
Test Limitations?
Impact of cultural bias?
Developmental level essential for accessibility?
Psychometrics
Psycho:
Metrics:
Psycho: Homan traits
Metrics: Measurement
Psychometric Properties:
standardised tests:
Reliability (Are the results replicable?)
Validity (Does it measure what it says it measures?)
Reliability:
3 Types
Test-Retest reliability
Inter-rater reliability
Intra-rater relability
Test-retest reliability:
Results are consistent when the same group is tested multiple times
Inter-rater reliability
Results are consistent when multiple raters (examiners) rate the same item
Intra-rater reliability
Results are consistent when the same person rates (examines) the test on more than one occasion
Validity: 2 Types
Content Validity
Construct Validity
Content Validity:
Test items represent all the possible test items that could be used to measure the construct that we want to measure
(tests have appropriate content that matches test purpose)
Construct Validity:
Test measures the construct that it is mean to be measuring
Reported as correlation co-efficient of the relationship between the test and the construct of interest
If the test correlates with other established tests that tests the same thing
(test vs existing tests)
Correlation coefficient:
0.1-0.25
Weak relationship
small correlation
Correlation coefficient: 0.3 - 0.49
Moderate relationship
medium correlation
Correlation coefficient: 0.5 - 1.0
Strong relationship
large correlation
Which correlation coefficient should you look for?
r= >0.5
Descriptive Statistics:
Measures? (3)
Summary / description observed for a given variable in a dataset
Measures central tendency
Measures variability/spread
Central tendency:
Tells us about the middle or centre of the data
Central tendency:
measures (3)
Mean: average (total ages / n)
Median: middle value when data is arranged in order
Mode: most commonly occurring value
Variability: measures (5)
Interquartile range
Range
Standard deviation
Skewness
Kurtosis
Variability: measures
Interquartile range:
Interquartile range - spread of middle half (25% above and below mean = 50%) of data
Variability: measures
range
Range - difference between highest & lowest
Variability: measures
Standard deviation
Standard deviation - measure of variance (interval or ratio data)
Low standard deviation: values closer to mean
High standard deviation: values spread @ wider range from mean
Variability: measures
Skewness
Skewness - imbalance of spread
/ : negative skew
\ : positive skew
Variability: measures
Kurtosis
Kurtosis - outliers
Inferential Statistics: predicts__
Predicts things about the population from a representative sample
Inferential Statistics: creates ___ distribution?
Standard normal distribution, bell curve
Norm referenced tests:
What are they?
Represents most tests SLPs use
Always standardised ( raw scores need to be converted to standard scores)
Norm referenced tests:
purpose
To compare a client’s results with the average
Norm referenced tests:
ICF relevance
Evaluates body structures & functions
Norm-referenced tests:
Positives (4)
Objective
Efficient
Widely recognised
Clear guidelines for administration & scoring
Norm-referenced tests:
Negatives (5)
Unable to be adapted
Does not represent real-life communication
Time bound, snapshot
Reflects response, but not the process of response (how client got there)
Content/ materials may not be appropriate to all clients
Criterion-referenced tests:
what are they? (2)
Evaluates what a client can or cannot do
Either standardised (scaled scores) or raw scores used
Criterion-referenced tests:
Compares client performance to: (3)
Level of ‘acceptable’ performance
Pre-determined criteria
Known developmental stages
Criterion-referenced tests:
ICF relevance?
Evaluates body structures & functions
Criterion-referenced tests:
Used to evaluate? (3)
Neurogenic communication disorders
Fluency
voice
Criterion-referenced tests:
Positives (4)
Objective
Efficient
Individualisation/ flexibility possible (for non-standardised tests)
Guidelines for administration and scoring (variable)
Criterion-referenced tests:
Negatives (2)
Does not replicate real life
Standardised criterion-referenced tests do not allow for individualisation
The assessment Manual:
You must know procedures (5)
- Use the Index to read the manual
- Check reliability & Validity (correlation coefficient)
- Is your client appropriate (age, language background, referral information)
- Follow administration procedures exactly as stated
- Find tables to turn raw scores to standardised scores
The assessment Manual:
Why do you need to know the procedures? (3)
Inaccurate administration = invalid results
Potentially harmful (client doesn’t get help or gets unnecessary help)
Ethics compromised
The assessment Manual:
Considerations (8)
Basal: starting point
Ceiling: ending point
Repetition: of items (might not be able to repeat
Cues & prompts: allowed?
How to assess/score: specific responses
Latency: time to respond
Feedback: allowed
What is each subtest measuring?
Test Scoring Tables & Appendices
Information: (5)
Raw score conversion to scaled score
Subtest scores
Core/index scores
Percentiles, confidence intervals
Suggested classification of scores (within normal distribution)
Standardised tests &
operational Error (5)
Time influence
Test content error (client has a different culture
Testing condition
Examiner error
Examinee error (client may be feeling sick)
Chronological age:
The exact age of a person in years, months and days
Raw score:
The initial score obtained based off of their correct or incorrect responses.
Not meaningful unless converted to other scores or ratings
Basal:
Starting point for tests administration & scoring
Ceiling:
Ending point of a test.
Typically determined by number of consecutively incorrect responses
Standard score
Standard score reflects performance compared to average and the normal distribution
Percentile rank:
95th percentile means
Tells the percentage of people scoring at or below a given score
95th percentile = only 5% of scores were higher than yours
You scored better than 95% of people who scored before you
Scaled Score
Reflects performance compared to the normative sample
However, these scores do not necessarily follow a normal distribution
Confidence Interval
Degree of certainty on the statistical values obtained are true
Takes into consideration human variability
Normal Distribution
Bell curve graph
Indicates values near the mean occur more frequently than the values away from the mean
