Measurement Reliability

Question 1

What is the difference between reliability & validity?

Answer 1

• Reliability: Stability of the measurement

- Validity: Meaningfulness of the measurement

Question 2

What are the different types of data?

Answer 2

• Continuous (e.g. TUG)
• Ordinal (e.g. stroke severity: mild, mod, severe)
• Nominal (i.e. categorical, e.g. soccer, AFL, rugby)
• Dichotomous (e.g. injured, not injured)

Question 3

What are the types of reliability?

Answer 3

• Inter-rater (between different assessors)
• Intra-rater (test-retest, within one assessor)
• Internal consistency (agreement between items that measure the same construct)

Question 4

What are the methods of measuring reliability?

Answer 4

• Kappa
• Bland-Altman plot
• Minimum detectable change
• Others e.g. intra-class correlation coefficient (ICC)

Question 5

What is the kappa statistic?

Answer 5

• 50% agreement on any yes/no question, e.g. 50% agreement if two people flip a coin
• Proportion of agreement beyond that expected by chance

Question 6

What is the formula for the kappa statistic?

Answer 6

k = (observed agreement - chance agreement) / (1 - chance agreement)

Question 7

What are the other types of kappa?

Answer 7

• Weighted: Gives point for partially correct answer (e.g. mild-mod)
• Prevalence-adjusted bias-adjusted kappa (PABAK): E.g. when yes/no split is 5:95% rather than 50:50

Question 8

What does method agreement consider?

Answer 8

How an old measurement device compares to a new measurement device

Question 9

What does the difference between the before & after treatment measurements represent?

Answer 9

• Error of measurement

- Genuine improvement

Question 10

What is the minimum detectable change?

Answer 10

• How much random noise/error is in a measurement

- Smaller random noise = change more easily detected

Question 11

What is the standard error of measurement (SEM)?

Answer 11

• Every measurement is made up of the true value +/- error

- SEM = Standard deviation of the error

Question 12

What does it mean when the difference between the pre-treatment score & post-treatment score is bigger than the MDC?

Answer 12

We are 95% confident a true change has occurred

Question 13

True or false:
An evidence-practice gap is, by definition, the gap in time which occurs between research evidence generation and when this evidence is routinely integrated into practice

Answer 13

False

Question 14

The evidence-to-practice pipeline highlights the steps and possible points of ‘evidence leakage’, involved in the path from the generation of research to its use in practice. A health professional who has trouble keeping up with the growing pool of information in their area of practice, is facing a barrier at which step in the pipeline?

Answer 14

Being aware of the evidence

Question 15

When may a social barrier in the implementation of evidence in practice occur?

Answer 15

When an expectation from a patient that a particular intervention will be provided

Question 16

In terms of the uptake of evidence-based practice, the comment “Using evidence-based techniques to guide my clinical practice would be good if it meant that I did not have to spend too much extra time reading journals”, could be attributed to an individual in which state of behaviour change?

Answer 16

Contemplation

Question 17

What is the theory of diffusion of innovations?

Answer 17

A theory which involves using role-models and social influence to help in the adoption of health practices