Traditional outcome measures Flashcards
What are the 3 main components for measuring quality of care according to Donabedian?
Structure –> Process –> Outcome
What is an outcome measure?
A measuring instrument that provides a rating or score (categorical or continuous) that is intended to represent some aspect of the patient’s medical status.
How does ISPOR divide outcome measures?
based on the way they are assessed
How are surgical outcomes traditionally measured?
LOS, complications, reoperations
Define surgical complications.
Any deviation from the normal postoperative course.
Explain limitations of current studies on complications descriptions.
• About 80% of studies describing complications failed to provide any information about their definition and severity.
Sometimes, papers published in very high IF journals did not specify duration of F/U, no outpatient information, not a single complication defined, no severity grade used.
Explain the advantages of using pre-defined time points for assessment of complications (as opposed to index hospitalization)
- Time points are important to assess post-op later outcomes/complications! Those are based on studies that have shown that mortality 1-year after surgery is mainly due to disease progression vs. surgery-related mortality.
o Surgery-related mortality curve flattens after 90 days.
o If assessment time point is too long, we might record complications that are unrelated to the surgery. With that 90-day time points we will not be “over-recording” complications.
How should complications be defined?
*We need criteria that are disease-specific, procedure-specific but also needs to be standardized in definition. A common criteria in definition is the severity of the complication.
Explain the Clavien Dindo classification
- Severity defined according to the degree of invasiveness of the treatment needed to correct the complication
- Grade of complication severity correlates with the cost of different complications
- The Dindo Clavien classification is the most widely used classification for complication severity
Name the limitations of the clavien-dindo classification system
o Considers only the single most severe complication experienced by the patient
o “Ignores” events of lesser severity; underestimates overall morbidity burden
o Uses an ordinal dichotomous scale = large sample sizes
Explain the Comprehensive complication index
• Accounts for all complications experienced by a patient: better reflect morbidity burden
• Calculated as the sum of all complications weighted for their severity. Weighed continuous score from 0-100.
o 0 = no complication
o 100 = Death of the patient
• Provides a continuous rather than ordinal/dichotomous measure of complications
• Increased sensitivity to detect differences in complications
• May decrease sample size requirements in comparative studies (d/t continuous scale)
How many CCI points are considered clinically significant?
10 pts
Name the limitations of the CCI
o Initial validation process focused on surgeries with high post-operative morbidity (e.g. esophagus, pancreas)
o Performance in less extensive procedures is still uncertain
- Distribution very skewed towards 0 –> hard to deal with statistically
Explain the limitations of LOS as a surgical outcome measure.
- Does not account for the late recovery stage
- Patient could be ready to be discharged but not be depending of the hospital culture, their willingness to be discharged, the surgeon’s preference and PO support at home.
What is the median difference between LOS and time to functional recovery ?
Median difference of 2 days between LOS (5 days) and time to functional recovery (3 days)
Differentiate LOS from TRD.
TRD has standardized criteria to allow for more accurate comparison of results.
How were the criteria for TRD determined?
Using Delphi method
Name the criteria for the TRD
- Tolerance of oral intake (at least 1 solid meal w/o N/V + should drink actively >800-1000ml/d w/o IV)
- Recovery of lower GI function (passed flatus)
- Adequate pain control with oral analgesia (pt should rest and mobilize (sit and walk) without sig pain (=<4 on pain scale).
- Ability to mobilize and self care (pt should be able to perform ADLs)
- Clinical exam, lab tests show no evidence of complications (normal oral temp, pulse, BP and resp rate, Hgb, able to empty bladder)
How was the TRD validated?
TRD consensus validated against multiple hypotheses (known groups):
- TRD is longer in subject undergoing open surgery
- TRD is longer in subjects having lower preoperative physical status (ASA > 3)
- TRD is longer in elderly subjects (>80 years old)
- TRD is longer in subjects having postoperative complications
- TRD is longer in subjects having emergency colorectal surgery
- TRD is shorter than actual hospital length of stay
What were the results of the TRD validation?
Results showed sig diff in TRD for all hypothesized groups except octogenarians were not sig diff from non-octogenarians.
Conclusion: Validity and reliability of TRD as a measure of short-term (to intermediate stage). Using TRD as an alternative to LOS may decrease sample size requirements in future RCTs.
Name the limitations of TRD.
- Resource intensive - need for prospective data collection and daily patient assessment
- Lack of validity in most surgical contexts
What is a composite outcome?
- A composite endpoint combines multiple components of interest into a single variable
- Patients who have experienced any of the events specified are considered to have experienced the composite outcome
What are the advantages of composite outcomes?
- Avoids the need to choose a single primary endpoint when many may be of equal importance
- Increasing number of events, reduces sample size requirement
What are the limitations of composite outcomes?
- Observed between-groups differences from a composite endpoint may be misleading or lead to a conclusion that all the components contribute equally to that difference
- Interpretation could be problematic when component endpoints are dissimilar in patients or clinicians importance
- Interpretation could be problematic if either the rates or relative risk reduction vary appreciably across components
Name an example of composite outcome
Textbook outcome
What is textbook outcome?
• TO represents the optimal course following surgery that is better aligned with patient expectations
around “optimal” care
• TO is used to assess quality or success of complex surgical procedures
• TO encompasses determinants of quality of care from different domains including structure, process, and individual outcomes
How is the evidence regarding PROMs?
There is still very limited evidence supporting what are ‘best practices’ when measuring PROMs in perioperative research and clinical settings.
Define “patient-centered care”.
Delivery of care that is respectful of and responsive to individual patient preferences, needs and values. (Many definitions are available in the literature, but this is the favorite one)
Define “shared decision-marking”.
An approach where clinicians and patients share the best available evidence when faced with the task of making decisions, and where patients are supported to consider options, to achieve informed preferences.
Name 3 interventions to improve surgery outcomes.
Prehabilitation, minimizing surgical trauma through minimally invasive surgery, ERAS programs
What are the types of PROMs?
- Unidimensional/ Multidimensional
* Generic/Condition-specific
What is a unidimensional PROM?
Questionnaires that measure a single construct.
May have one single question (item) or multiple questions (items)
Produces one single score representing the construct of interest
Provide examples of unidimensional PROMs.
Visual analogue scale (Pain), Duke activity status index (DASI), Fatigue severity scale (FSS)
What are the pros and cons of unidimensional PROMs?
Pros: Usually short (low response burden)
Cons: May not provide a complete picture of the construct being measures
What is a multidimensional PROM?
Questionnaires that measure multiple constructs
Always have multiple questions (items)
Provide a profile of scores (each score represents a different domain of health)
Provide examples of multidimensional PROMs.
SF-36 (physical and mental summary scores), PROMIS-29 (Domains: Anxiety, depression, fatigue, pain, physical function, sleep, social roles), WHODAS-2.0 (Domains: Cognition, mobility, self-care, getting along, life and participation).
What are the pros and cons of multidimensional PROMs?
Pros: Provides a more complete picture of the construct being measured.
Cons: Usually longer (higher response burden)
What is a generic PROM?
Measures intended to address general aspects of health that are not specific to a particular disease or condition
Provide examples of generic PROMs.
SF-36, WHODAS 2.0, EQ-5D
What are the pros and cons of generic PROMs?
Pros: Can be used to compare health status across different patients populations and to population norms.
Cons: Not suited to detect changes in health issues that are condition-specific.
What is a condition-specific PROM?
Measures intended to address aspects of health that are impacted by a specific disease or condition
Provide examples of condition-specific PROMs.
Quality of recovery questionnaires (QOR-9, -15, -40), Abdominal Surgery Impact Scale (ASIS)
What are the pros and cons of condition-specific PROMs?
Pros: Can detect health issues that are condition-specific
Cons: Not suited to compare health status across different patient populations and to population norms
Describe what is the recovery period and the downstream period.
• Recovery period
o Focused on how patients’ respond to the physiological stress imposed by the surgical procedure
o Can be divided into early, intermediate and late recovery
o Assessed within 3 months after surgery, time after which most patients are recovered back to preoperative health status or above.
• Downstream period
o Focused on the impact of the surgical procedure on the health issue that it was meant to treat
o May detect long-term/permanent sequalae of the surgical procedure (i.e. new disability)
o Assessed beyond 3 months after surgery
Define “measurement property”
The ability of an instrument to truthfully and comprehensively measure a specific construct and demonstrate that it is discriminative, sensitive and reliable.
What were the questions asked by Rajabiyazdi et al. about recovery to patients?
- What does recovery from surgery mean to you?
- Overall, do you feel that you are already completely recovered (i.e., back to normal)?
- What made you consider yourself to be completely recovered?
- When will you consider yourself to be completely recovered?
- What is the most important sign of recovery for you?
What were the 5 themes covered by the Rajabiyazdi study re: recovery meaning to patients?
• Returning to habits and routines • Overcoming mental strains • Regaining independence • Resolution of symptoms • Enjoying life *Sub-themes are also developed.
What are the advantages of measuring recovery using PROMs?
- Allow a broad assessment of health status across different domains of health
- Engage patients as the key stakeholders in the recovery process
- Questionnaires can be completed at different time points, allowing a better understanding of the recovery trajectory
What is the evidence on the impact of interventions aimed to improve recovery on PROMs
Multiple studies show no impact or inconsistent effect of ERAS and prehab on PROMs despite them having a positive effect on standard measures such as LOS and complications
Conclusion: PROMs show inconsistent/negative results for interventions! This could mean 2 things:
- These interventions (e.g. prehab, ERAS), have fewer advantages than expected
- There is a lack of adequate PROMs to assess recovery after surgery
Define reliability
The degree to which the measurement is free from measurement error
Define interrater reliability
The proportion of the total variance in the measurements which is due to the “true” differences between patients
Define internal consistency
The degree of the interrelatedness among the items
Define measurement error
The systematic and random error of a patient’s score that is not attributed to true changes in the construct to be measured.
Define validity
The degree to which an HR-PRO instrument measures the construct(s) it purports to measure
Define content validity
The degree to which the content of an HR-PRO instrument is an adequate reflection of the construct to be measured
Define construct validity
The degree to which the scores of an HR-PRO instrument are consistent with hypotheses (for instance, with regards to internal relationships, relationships to scores of other instruments, or differences between relevant groups) based on the assumption that the GR-PRO instrument validly measures the construct to be measured.
Define criterion validity
The degree to which the scores of an HR-PRO instrument are an adequate reflection of a “gold standard”.
Define responsiveness
The ability of an HR-PRO instrument to detect change over time in the construct to be measured
What are the minimum standards for PROs in comparative effectiveness research?
- Content validity
- Reliability (including internal consistency)
- Construct validity
- Responsiveness
Where is the state of evidence regarding PROMs?
Current PROMs to assess recovery after abdominal surgery:
- Were NOT developed according to optimal scientific standards
- Have limited evidence supporting their measurements properties
- Even tools widely used in the literature have limited measurement properties*
What is the method used by Dr Fiore to develop a new PROM tool?
Rasch Analysis (modern psychometric method)
What are the areas covered in Dr. Fiore’s PROM questionnaire
- Pain
- Emotions
- Sleep
- Skin repair and skin sensations
- Energy and drive
- Visceral function
- Activity participation
- Physical function
Name the advantages of SF-36
- Generally familiar to researchers and clinicians
- as a multidimensional mesure, it can assess the impact of surgery on multiple levels of health
- Evidence supports its construct validity and responsiveness in the context of late recovery
Name the disadvantages of SF-36
- The recall period (4 weeks) may be too long to detect rapid changes in post-op health status
- As a generic measure, it does not cover important domains of health that are relevant to surgical patients (e.g. recovery of GI function, wound healing).
Classify the SF-36 in terms of PRO categories.
multi-dimensional and generic
Name the advantages of QoR-15
- Measure specifically developed for surgical recovery
- The recall period (24 hours) seems adequate to detect rapid changes in postop health status (early and intermediate recovery)
- Previous research supports adequate reliability, construct validity and responsiveness
Name the disadvantages of QoR-15
- Evidence of content validity is poor
- Was designed as a unidimensional measure, but shouldnt be because questions relate to different domains.
Classify the QoR-15 in terms of PRO categories.
Uni-dimensional because produces a single score, but technically should be multi-dimensional. Condition-specific because specific to surgical patients recovering from sx
Name the advantages of PROMIS-29
- Very popular among clinicians and researchers
- Multidimensional
- This measure was developed using modern psychometric methods (item-response theory) = continuous measure, more sensitive to change.
Name the disadvantages of PROMIS-29
- The recall period (7 days) may be too long to detect rapid changes in post op health status
- Generic
No evidence supporting its measurement properties in surgical recovery context
Classify the PROMIS-29 in terms of PRO categories.
Multidimensional and generic
Name the advantages of ASIS
- Measure specifically developed to measure surgical recovery after abdominal surgery
- Multidimensional
- Short recall period (24h)
- Previous research supports adequate reliability, content validity, and responsiveness
Name the disadvantages of ASIS
- Rarely used
- Recent research does not support its construct validity (ability to differentiate groups with different recovery trajectories
Classify the ASIS in terms of PRO categories.
Multidimensional and condition-specific
Name the advantages of VAS
- Familiar to researchers and clinicians
- An adequate recall period can be set to detect rapid changes in post-op health status
- Low response burden (only one question)
Name the disadvantages of VAS
- Not much research assessing its measurement properties in the context of post-op care.
Classify the VAS in terms of PRO categories.
Unidimensional and generic
Name the advantages of WHODAS-2.0
- multidimensional
- This measure was developed using modern psychometric methods (item-response theory) = continuous measure, more sensitive to change
- Evidence supports its construct validity and responsiveness in the context of late recovery.
Name the disadvantages of WHODAS-2.0
- The recall period (30 days) may be too long to detect rapid changes in post op health status
- Generic
Classify the WHODAS-2.0 in terms of PRO categories.
Multidimensional and generic
