reliability and validity

Question 1

What is test-retest reliability

Answer 1

Test-retest reliability is commonly estimated by calculating the correlation coefficient of the measured values at two separate time points. A higher correlation between the values of the two test occasions indicates greater temporal stability or test-retest reliability. Cicchetti (1994) defined 0.4 to 0.59 as fair, 0.60 to 0.74 as good, and above 0.75 as excellent/

Question 2

importance of test-retest reliability

Answer 2

ensures that the measurements obtained in one sitting are both representative and stable over time.

Question 3

what is internal consistency

Answer 3

a measure of how well a test addresses different constructs and delivers reliable scores.80 or greater is generally considered good internal consistency

Question 4

why is internal consistency important

Answer 4

when researchers want to ensure that they have included a sufficient number of items to capture the concept adequately.

Question 5

content validity

Answer 5

Content validity evaluates how well an instrument (like a test) covers all relevant parts of the construct it aims to measure.

Question 6

importance content validity

Answer 6

good indicator of whether the desired outcome is being measured. in this case, this would be the functional ability of a patient.

Question 7

what is responsiveness

Answer 7

The ability of an outcome measure to detect changes over time in the construct to be measured.

Question 8

importance of Responsiveness

Answer 8

Important to see if patients are progressing or not with rehabilitation.

Question 9

minimal detectable change

Answer 9

An MDC at 90% confidence level indicates that there is only a 10% chance that a change above this threshold could be due to chance variability. For LEFS the MDC at 90% confidence level is 6, this means that true change has occured if the score changes by 6.

Question 10

minimal clinically important difference

Answer 10

MDIC= 9, this shoes that for a change on the LEFS to be significant it has to be over 9 points.

Question 11

What are the cosmin guidlines

Answer 11

The COSMIN checklist (COnsensus-based Standards for the selection of health status Measurement INstruments) was developed in an international Delphi study to evaluate the methodological quality of studies on measurement properties of health-related patient reported outcomes.

Question 12

How would you use it to set functional goals- Binkley study?

Answer 12

• Binkley et al stated goals could be set based on improving a scored- however, this was set based on research in an orthopaedic setting. Within a setting like DMRC, it is unrealistic to expect clinically meaningful change as you will only be targeting specific aspects of the LEFS.
• With some of the patietns here at DMRC I have used the LEFS to monitor changes in gait (for example distance) and also the ability to ascend and descend staris.

Question 13

How was content validity deemed inconsistent

Answer 13

In accordance with the COSMIN guidelines, the content validity of the LEFS was then rated subjectively by the reviewers. The results were obtained from evaluating 7 articles that assessed measurement properties. ICF categories d4 mobility (e.g. movement with equipment and
using transportation such as a bike or public transport)
and d5 self-care (e.g. toileting and caring for body parts)
may not be sufciently covered

Question 14

what is ICF

Answer 14

International Classification of Functioning, Disability and Health

Question 15

How is the effectiveness of treatment measured within study

Answer 15

Personal experience with patient- how did LEFS inform treatment

Question 16

Personal experience with patient- how did LEFS inform treatment

Answer 16

• Highlighted deficits in walking 2 blocks/mile and stair- complete gait rehabilitation and stairs
• Patient d/c stairs- working on lots of different areas- neglected stairs as able to mobilise around DMRC using lift-

Question 17

Why is it not very sensitive to small amounts of change?

Answer 17

Here at DMRC it is unrealistic to expect large amounts of variance as you will only be working on specific aspects of the LEFS. To measure more specific changes at certain joints, joint specific PROM may be better

Question 18

0-4

Answer 18

Extreme Difficulty or Unable to Perform Activity
Quite a Bit of Difficulty
Moderate Difficulty
A Little Bit of Difficulty
No Difficulty

Question 19

Why have you chosen this OM

Answer 19

• I have chosen this outcome measure because one of the main aims of physiotherapy is to improve a patient’s functional ability, and this is something patients themselves strive towards. Therefore, it is important that there is an outcome measure that can monitor how effective physiotherapy treatment is at improving patients’ function.
• For example, whilst on placement at DMRC I treated 4 patients all with LL conditions, which impacted on their functional ability. Following on from discussion with regards to patients main aims of physiotherapy intervention, it became clear that improving function was a priority. Therefore, I used this OM as a measure of this. With the majority of the patients I treated whilst on placement were not expected to see “clinically important difference” due to the fact it was unrealistic to expect changes in every aspect of the LEFS, as their injuries were complex in nature. Therefore, I used the LEFS to monitor changes in specific aspects such as stairs ability, walking and daily functional ability.
• Furthermore, research by Belete et al (2021) into the impact of LL fractures on functional ability, used the LEFS to measure this. The results showed that 80% of patients had functional deficits, showing how prevalent of a problem this is. They concluded that physiotherapist should consider early physiotherapy intervention, and appropriate functional limitation reduction strategies.
• PROM have a vital place in clinical practice- they allow for physios to measure, compare and improve a care system= provide better care at patient level
• PROM- can lead to patient centred care

Question 20

How can LEFS begin to aid discharge

Answer 20

• The first stage of discharge planning is to highlight impairments early. It can be used to identify functional defects early in an acute setting, by focusing on patients ADLs. In order for patients to be discharged they need to be competent in completing their ADLs, therefore LEFS can be used to highlight areas of ADLs in which patients may be less confident in, which can tailor treatment methods.
• Furthermore, by addressing the defects in functional ability early, it can reduce the amount of time a patient spends in the hospital. This can reduce the stress on individual trust by freeing up more hospital beds.
• Furthermore, if patients are happy with all of their ADL, that are included in LEFS, then they are less likely to be re admitted- for example on a orthopaedic word
  *More research is necessary into this. However, to achieve this this OM should not be used in isolation, could be used alongside PPE 15- 15 questions assessing- assessing 8 key
  aspect of care: Information and Education;
  Co-ordination of Care; Physical Comfort; Emotional
  Support; Respect for Patient Preferences; Involvement of
  Family and Friends; Continuity and Transition
• research by Turncotte et al, 2022 suggests LEFS is a useful tool for aiding clinical resource allocation, and it can be used to predict outcomes= e.g. quick D/C.

Question 21

Link between data collected on OM and service

Answer 21

• It can be used to evaluate service effectiveness by monitoring progressions in patients’ lower limb function. Service effectiveness= devine outcomes to be achieved- select right measures- monitor/evaluate effectiveness- look at continuous improvement.
• This in turn, can have an influence on the number of staff and the number of patients the service provides. For example, it can be used to measure functional changes over a period of time, lack of change in multiple patients could show a lack of treatment time? Non-effective rehabilitation? Too many patients? if there is limited improvement observed within a patient population then it could be indicating a lack of staff, too many patients or a lack of effective treatment.
• Difference services can also be reviewed by looking at the results from the LEFS, due to its ability to be applied in multiple different areas.
• The LEFS can provide useful information for this through patients interpretations of the LL function, but clinicians should use objective outcome measures alongside this.
• Impact on national guidelines: NICE guidelines are reviewed frequently, they are reviewed by looking into service effectiveness of specific trusts.
• Impact guidline has on cost is then considered- so LEFS results could be monitored for specific services and if improvements are not being made, could have implication on increased funding being made available.

Question 22

Why use objective OM alongside

Answer 22

• Patient-reported outcome measure is limited due to the fact is a patients perception of their functional ability, and although this is beneficial for clinicians to know, to measure a true change in function objective measures should be used.
• For example, HiMAT can be used alongside it. The High Level Mobility and Assessment tool (HiMAT) is a standardised outcome measure used to quantify motor performance in individuals with high-level balance and mobility deficits. Or for gait 6MWT, TUG.
• By the nature of being subjective, PROMs may be affected by internal factors, such as mood, expectations, time and sentiments, and external factors such as treatment context, interactions with the healthcare providers and patients’ socioeconomic situation, which leads to fluctuations in the outcomes.
• Limitations of PROM
  o Moreover, patients are not necessarily able to identify an improvement in their own health.
  o Therefore, it is important to combine PROMs with functional outcomes in order to gain insight into both physiological effect and patients’ well-being

Question 23

Which measures are used in practice; what could be used in practice?

Answer 23

• SF 36- research comparing LEFS TO SF-36 which are both PROM, The LEFS is reliable, and construct validity was supported by comparison with the SF-36. The sensitivity to change of the LEFS was superior to that of the SF-36 in this population. The LEFS is efficient to administer and score and is applicable for research purposes and clinical decision making for individual patients.

Question 24

How might this impact decision-making: in either the short-term or longer-term?

Answer 24

• Short term- if you are working a lot in physiotherapy sessions on a functional task such as completing the stairs, and as a therapist you are making physical progressions. But the patient reports no changes in ability, it could highlight a potential psychological issue that could be addressed through education.
• Long term- service effectiveness. Lack of progress= change rehabilitation techniques- try different exercises? Education?

Question 25

How does the outcome measure provide clinically useful information?

Answer 25

• By providing patients perception of their functional ability.
• Can identify deficits in a patients confidence when completing a specific activity.
• Can be used to monitor the effectiveness of interventions

Question 26

SRM calculated

Answer 26

is an effect size index used to gauge the responsiveness of outcome measures, calculated by dividing the mean change by the standard deviation of the change.

Question 27

what is ICC

Answer 27

It describes how strongly results resemble themselves

Question 28

what is Cronbach’s alpha

Answer 28

shows how closely related a set of items are

Question 29

final key point

Answer 29

• reliable and valid tool to measure functional ability
• Can impact on commissioning of service- rehab effectiveness, contact time, patient load- which in turn could lead to funding
• Used alongside other OM due to limitations of PROM