Measurement Reliability Flashcards
What is the difference between reliability & validity?
- Reliability: Stability of the measurement
- Validity: Meaningfulness of the measurement
What are the different types of data?
- Continuous (e.g. TUG)
- Ordinal (e.g. stroke severity: mild, mod, severe)
- Nominal (i.e. categorical, e.g. soccer, AFL, rugby)
- Dichotomous (e.g. injured, not injured)
What are the types of reliability?
- Inter-rater (between different assessors)
- Intra-rater (test-retest, within one assessor)
- Internal consistency (agreement between items that measure the same construct)
What are the methods of measuring reliability?
- Kappa
- Bland-Altman plot
- Minimum detectable change
- Others e.g. intra-class correlation coefficient (ICC)
What is the kappa statistic?
- 50% agreement on any yes/no question, e.g. 50% agreement if two people flip a coin
- Proportion of agreement beyond that expected by chance
What is the formula for the kappa statistic?
k = (observed agreement - chance agreement) / (1 - chance agreement)
What are the other types of kappa?
- Weighted: Gives point for partially correct answer (e.g. mild-mod)
- Prevalence-adjusted bias-adjusted kappa (PABAK): E.g. when yes/no split is 5:95% rather than 50:50
What does method agreement consider?
How an old measurement device compares to a new measurement device
What does the difference between the before & after treatment measurements represent?
- Error of measurement
- Genuine improvement
What is the minimum detectable change?
- How much random noise/error is in a measurement
- Smaller random noise = change more easily detected
What is the standard error of measurement (SEM)?
- Every measurement is made up of the true value +/- error
- SEM = Standard deviation of the error
What does it mean when the difference between the pre-treatment score & post-treatment score is bigger than the MDC?
We are 95% confident a true change has occurred
True or false:
An evidence-practice gap is, by definition, the gap in time which occurs between research evidence generation and when this evidence is routinely integrated into practice
False
The evidence-to-practice pipeline highlights the steps and possible points of ‘evidence leakage’, involved in the path from the generation of research to its use in practice. A health professional who has trouble keeping up with the growing pool of information in their area of practice, is facing a barrier at which step in the pipeline?
Being aware of the evidence
When may a social barrier in the implementation of evidence in practice occur?
When an expectation from a patient that a particular intervention will be provided
In terms of the uptake of evidence-based practice, the comment “Using evidence-based techniques to guide my clinical practice would be good if it meant that I did not have to spend too much extra time reading journals”, could be attributed to an individual in which state of behaviour change?
Contemplation
What is the theory of diffusion of innovations?
A theory which involves using role-models and social influence to help in the adoption of health practices
