Exam 2 Flashcards
What is validity?
Truthfulness, meaningfulness, usefulness, and/or accuracy of study results
External vs Internal Validity
External Validity: generalizability of results
Internal Validity: controlled by the study design (blinding, instrumentation, attrition)
What is face validity?
Does a specific measure actually measure what it is designed to measure
What is content validity?
Does the measure represent all constructs of the measure (does it take all things into account)
What is concurrent validity?
Comparing your intervention to the Gold Standard
What is predictive validity?
Can it be used to predict a future score/outcome
What is construct validity?
How well the measure captures a defined entity (theoretical construct)
What is convergent validity?
Examines the degree to which the operationalization is similar to other operations that is should be similar to (one head start compared to others)
What is discriminant validity?
Examines the degree to which one thing differs from others (one head start compared to non-head starts)
How is validity typically measured?
Correlations, -1 to 1
What analyses are used for which data types?
Interval and Ratio (Continuous) - Pearson
Ordinal - Spearman Rank
Nominal (Dichotomous) - Phi
What is Reliability?
Consistency of a specific measure
Ability to produce consistent repeated measures of a test
What are the two components of reliability
True Component Error Component (variety of sources)
What type of data is required for Reliability Measures?
Continuous - Ratio or Interval
What are the breakdown scores for ICC (Reliability)?
Good: > .75
Moderate: .51 - .75
Poor: < .50
What type of data is agreement?
Categorical (Nominal)
Kappa statistic takes out the chance aspect
What are the Kappa score breakdowns?
Almost Perfect: .81 - 1.0 Substantial: .61 - .80 Moderate: .41 - .60 Fair: .21 - .40 Slight: .01 - .20 Poor (equal to chance): < 0
What is the Minimal Detectable Change?
Smallest amount of change an instrument can accurately measure
Changes must exceed MDC to be beyond measurement error
Does not provide clinical meaningfulness
What is the Minimal Clinically Important Difference?
Smallest difference that clinicians and patients would care about
Identify change in health status measure associated with improvement that is meaningful
Compares two measures (Pain: VAS, clinician-derived measure: ROM)
What is Ceiling Effect?
Instrument does not register a further increase in score for higher scoring individuals
What is Floor Effect?
Instrument does not register a further decrease in score for lower scoring individuals
What variables are Statistically significant?
p-values
Precision of estimation/confidence intervals
Type 1 and Type 2 errors
Power
What variables are Clinically significant?
Size of the difference
Does change exceed MCID
Effect Size measurements
Specificity, sensitivity, LR, NNT, RR, ARR
p-values
Risk of Type 1 error
Does not indicate importance or clinical relevance
What are type 1 errors?
Reject the null hypothesis when it is actually true
Conclude a difference exists, but it doesn’t actually exist
Rare (False Positive)
What are Type 2 errors?
Do not reject null hypothesis when it is actually false No significant difference detected, but a difference exists More Common (False Negative)
What factors impact statistical power?
Significance (a)
Effect size (differences between measures and variance)
Sample size
How does sample size impact power?
Larger sample increases the ability to detect smaller differences between groups
What is effect size?
Determines magnitude of treatment effect (meaningfulness of results)
Allows normalized comparison of results (removes units from outcomes)
Accounts for variation across samples
What is Cohen’s d?
Most common way to express effect size
Usually positive, negative indicates a decrease (pain)
What are the breakdowns for effect size scores?
Large: .80
Moderate: .50
Small: .20
How are SEM, MDC, and MCID related?
SEM and MDC provide context, but MCID provides meaning
What errors influence statistical power?
Type II Errors
Sample size and variance
Sensitivity =
a/(a + c)
SnOut
Specificity =
d/(b + d)
SpIn
+LR =
Sensitivity/(1 - Specificity)
-LR =
(1 - Sensitivity)/Specificity
PPV =
a/(a + b)
NPV =
d/(c + d)
What are Likelihood Ratios?
Incorporate sensitivity and specificity
Provide a direct estimate of how much a test result will change the odds of having that condition
What are the LR breakdowns?
Strong (conclusive): +LR >10/-LR < .1
Moderate (important): +LR 5 - 10/-LR .1 - .2
What is the order of the Diagnostic Process?
Pre-test probability (prevalence)
Patient History (develop working hypothesis)
Select specific tests to confirm/refute
Post-test probability (treatment threshold: likelihood patient has that disorder)
What is the PEDro scale?
Allows quantification of the quality of a research study
10-point scale (11 questions)
Designed for clinical trials, but may be used for other types of studies
Disease-oriented vs patient-oriented outcome measures
Disease-oriented: Physiology of illness (ROM)
Patient-oriented: direct patient interest, patient-oriented evidence that matters (POEMs), functional aspects of the loss of ROM
Clinician-derived vs patient self-report outcome measures
Clinician-derived: almost always disease-oriented (MMT, ROM)
Patient self-report: general or global health, survey patient fills out
What is a disability?
Inability/Limitation in performing socially defined roles/tasks expected of an individual within a sociocultural and physical environment due to functional limitations
Where is the paradigm shift in measuring outcomes moving towards?
Not only measuring impairments, but also quantifying changes in: functional limitations, disability, and QOL
What are the Objective Outcome Measures?
ROM, MMT, Limb Girth, Blood Count
What are the Subjective Outcome Measures?
Self-Report Questionnaires (by patient or clinician)
These focus on functional limitations, disability, or QOL
Disparity between patient and clinician-reported (clinicians rate higher)
What are Global Health Measures?
Lean towards indicators of disability and QOL
Good at tracking patients with chronic diseases (limited ability for active populations: ceiling effect)
Examples: SF-36 and SF-12, Global Rating of Change, Sickness Impact Profile
Pros and Cons of SF-36
Low scores indicative of greater disability
Pros:
You can give it everyone
Helps you refer to other health professions
Cons:
Not specific
Embarrassing to answer
May answer how they think PT wants them to
Region-Specific Health Questionnaires
Scales to specifically look at body-region of interest
Focus more on functional limitations and disability
Good at tracking recovery from specific pathologies
Better utility for active populations
Examples: FAAM, DASH, LEFS
Can use it at baseline, during, and after for tracking
Oswestry Low Back Disability Index
Measure patient’s impairment and QOL in relation to LBP
10 questions
Higher scores = greater disability
Disabilities of the Arm, Shoulder, and Hands (DASH)
30 item, self-report to measure physical function and symptoms of musculoskeletal disorders of upper limb
Higher score = greater disability
Lower Extremity Functional Scale (LEFS)
Intended for use on adults with lower extremity conditions
20 items, 5-point scale
Higher scores = better function
Dimension Specific Health Questionnaires
More focus on psychosocial
Assess specific physical or emotional phenomenon (pain, anxiety, depression
Needs to be valid for population
Examples: Beck depression index, Pain disability Index, McGill Pain Questionnaire
Single Item Outcome Measures
Single Assessment Numeric Evaluation (SANE): rate current level of function for ADLs compared to prior level of function
Unidimensional
Too vague: not anchored directly to particular injury
What are the limitations of General Health Outcome Tools?
Not population specific
Physically active populations may be always seen as being “relatively” healthy compared to the whole population
How to pick the best outcome measure?
Measure should match the purpose
Able to discriminate among patients (validity)
Capacity to assess change over time (reliability, MDC, MCID)
What is measured value?
True Value + Error
Smaller error provides better indication of true value
Increased confidence in measure value
What is the Standard Error of Measurement?
Absolute reliability (typical error of individual score) Quantifies consistency of value in same digits as the measure Provides insight into meaningful changes
What are the 2 components of MCID?
Anchor-based: Linked to external anchor (global rating of change), dependent on recall and subject to bias
Distribution-based: based on statistical characteristics of the sample population (change beyond chance)l doesn’t account for patient perspective
Strengths of MCID
Threshold to detect change Accounts for patient perspective Set treatment goals (clinical) Determine sample size (research) Demonstrate treatment effectiveness
Limitations of MCID
Not universal fixed attribute (threshold without ranges)
Calculation methods vary (produce range of results)
Not transferable across patient populations (MCID ranges may vary)
What is a single subject design?
Prospective, extended baseline, controlled conditions
What is a case report (clinical case report)?
Description of clinical practice, non-experimental
Prospective or Retrospective (may be easier since you know more about them, highlights importance of documentation and outcome measures)
What is a case study?
Qualitative design, experimental
What is the hierarchy of Clinical Research Design?
Meta-Analysis Systematic Review RCT Cohort Study Outcomes Studies Case Control Study Cross Sectional Study Case Series Case Report
What are the purposes of a Case Report?
Share Clinical Experiences Illustrating EBP Develop Hypotheses for Research Build Problem-Solving Skills Test Theory Persuade and Motivate Help Develop Practice Guidelines and Pathways
What are the impacts of Case Reports?
Change clinical practice
Highlight unique patient presentation/diagnosis (special tests, imaging, metrics related to diagnostic accuracy [Sn, Sp, LRs])
Framework for treatment
Suggest areas for further research
What are the limitations to sensitivity and specificity?
Not clinically intuitive
Don’t change probability
What control measurement techniques are used in Case Reports?
Clinical
Functional
Patient-reported Function
What is the generalizability of Case Reports?
Most applicable to patient care (used in patient care, brings in the aspect of patient experience, not a controlled environment)
Least rigorous approach (sacrifice internal validity from confounding factors and smaller details)
Single person not representative of population (everyone is different, so RCTs and case reports can’t be applied in both ways to each person)
What is the quality of Case Reports?
Quality guidelines don’t exist
Many journals have suggested guidelines
ICF model may provide some structure
What is the format of a Case Report?
Intro (review of relevant literature, purpose statement)
Methods/Case Description
Results
Discussion and Conclusion
What makes up the intro portion of a case report?
Relevant literature review Patient condition Rationale for intervention and outcomes measures Indicate knowledge gap Purpose Statement Convince reader the topic is important
What makes up the Methods/Case Description portion of a case report?
Describe the patient:
Demographics
Past Medical History
Unique Presentation/Diagnosis
Examination Data:
Include reliability and validity
Rationale for selection
Describe clinical decision-making process (evaluation):
Treatment Approach
What makes up the results portion of a case report?
Describe the outcomes (each follow up point)
Consider table, graph, flow chart (chronological order)
Provide the facts (interpretation of findings reserved for discussion)
What makes up the discussion and conclusion portion of a case report?
Provide context to the results (meaning and application, related to MDC and MCID)
Compare and contrast to existing literature (relate to intro section)
Make specific recommendation to advance clinical practice (future research)
Clinical relevance
What should you be careful about in the discussion and conclusion section?
Don’t overgeneralize results
Can’t determine cause and effect (usually implied) since there is no control group
Does case report data have to be done consecutively?
No, one person can be from January - March, then another from July - September
Why is it important to track clinical outcomes?
Identify patterns
Insurance companies pay for outcomes
Treatments that work better/worse for future reference
Determine clinician effectiveness
What is Fee-for-Service?
Clinicians are paid based on the volume of services, not value
What is the Merit-based incentive payment system?
Performance-based payment adjusted to Medicare payment
Aggregate score across 4 categories determines payment adjustment (Quality, cost, advancing care info, improvement)
Unfortunately, many things not in your control
What is a Risk Adjustment?
Accounts for variables that influence outcomes (age, acuity, comorbidities, medication use)
Set reasonable goals
Helps predict number of visits required to achieve predicted outcome
What’s the purpose of benchmark data?
Effectiveness
Efficiency
Patient satisfaction
What is Evidence-based practice?
Integration of the best research evidence with clinical experience and patient values to make clinical decisions
Why do we need EBP?
Most clinical practices are handed down without ample scientific evidence demonstrating clinical efficacy
Need to be accountable for the care we render
EBP = accountability (patients, employers, insurance)
Link theory to practice (we don’t know if treating hamstring strains with US followed by flexibility exercises makes a difference in their outcome)
Why is EBP important?
Patient care improved
Third-party reimbursement
Development of knowledge base
Development and maintenance of respect within the healthcare community
What is evidence based practice not?
Cookbook/blueprint for practice
Conspiracy to discount what clinicians have been previously taught
Shouldn’t replace clinical judgment
Shouldn’t restrict practice
What are the Core dimensions of expert practice in PT?
Knowledge
Clinical Reasoning
Movement
Virtues
Clinical practice is:
Challenging Complex and uncertain Constantly changing and patient centered Demands innovation and creativity Perfect venue for the development of clinical reasoning abilities
What is clinical reasoning?
The sum of the thinking and decision-making processes associated with clinical practice
What is critical thinking?
One cognitive component of clinical reasoning where you analyze the evidence that exists in the literature, but it doesn’t encompass the contextual factors (patient and environmental factors from ICF framework) that are important in the reasoning process
What is clinical decision making?
The action taken upon thoughts (you can take actions without any thought)
Clinical reasoning:
is a continuous reflective activity
Engages the patient and family in a collaborative decision-making process
Acknowledges key contextual factors that will impact clinical interventions and patient outcomes
What are the 3 types of reflection in clinical reasoning?
Reflection-on-action: reflect after the experience occurs (novice)
Reflection-in-action: making changes during real time
Reflection-for-action: anticipates change for future interaction
What are the 2 types of Clinical Reasoning?
Hypthetico-Deductive Reasoning
Pattern Recognition Reasoning
What is Hypothetico-Deductive Reasoning
Generation of a hypothesis based upon results of tests and measure and then testing of that hypothesis (novices in all situations, and experts in challenging/unfamiliar cases when pattern recognition isn’t working)
What is Pattern Recognition Reasoning?
Quick retrieval of info from a well-structures knowledge base contrived upon previous experience (used by experts during familiar situations)
Deductive Reasoning:
Uses a structure to guide the thinking and action process
Emphasizes hypothesis generation based upon info gathered from the patient
Additional info is collected and interpreted in a cyclical process to disprove or confirm a hypothesis or diagnosis
Examples: ICF, Patient Management Model, CPG, CPR, Algorithms
Inductive Reasoning:
Requires previous experience with patients to “paint a picture”
Typically involves recognition of familiar clinical patterns and identification of the patient-specific contextual factors
Need experience to form the patterns
Examples: Reflection, Open-ended questions, Writing a story/clinical narrative
Teaching strategies for clinical reasoning:
Emotion and motivation are integral, not just cognitive aspects
Affective and behavioral factors
Use narratives to tell meaningful patient stories
What are the 3 categories of grading for clinical reasoning assessment tools?
Content knowledge
Procedural knowledge and psychomotor skills
Conceptual reasoning
What are the breakdowns for the Dreyfus and Dreyfus model of skill development?
Novice: Rule driven, analytical thinking, unable to prioritize info
Adv Begin: Begins determining relevant info, analytical thinking and pattern recognition, can generalize info
Competent: Emotion improves level of responsibility, clinical reasoning = more pattern recognition, sees big picture
Proficient: Clinical problem solving seems intuitive, comfortable with evolving situations, can live with ambiguity (uncertainty)
Expert: Open to notice the unexpected, perceptive in realizing what didn’t fit a pattern
What are the errors in clinical reasoning?
Lack of knowledge or faulty knowledge
Faulty data gathering process
Inability to interpret results of tests appropriately
What are cognitive biases of clinical reasoning?
Anchoring bias and premature closure (stop search early after explanation) Affective bias (emotion-based deviance from rational judgement) Availability bias (dominant recall of recent/common cases) Context bias (contextual factors that mislead)
What is the correlate between clinical experience and reflection? (Novice to Expert)
Reflection-on-action
Reflection-in-action
Reflection-for-action
In a methodological critique, what aspects have to do with quantitative research?
Internal validity
Reliability
Validity
External Validity
In a Methodological critique, what aspects are qualitative research?
Credibility (prolonged engagement and member checking)
Dependability (low inference methods and external audits)
Confirmability (triangulation and negative case analysis)
Transferability (generalize to theory)
What to look for in a results section?
Underlying conceptual/theoretical framework Evidence of themes/categories Clarity of findings Evidence of triangulation Data in support of findings
What to look for in a methods section?
Sample (linked with purpose)
Demographics
Data collection process that fits the purpose
Research design
Data analysis process (evidence of data reduction)
Linking of analysis to credibility factors
Ethics
What to look for in a Discussion section?
Place findings in context of the literature
Soundness of conceptual or theoretical argument
Knowledge of study limitations
Implications consistent with depth of data
Link to practice
What are the key concepts of Data Analysis?
Concurrent with data collection
Involves large amounts of narrative data
Primarily inductive
Final results is always a higher level of synthesis
Includes:
Coding, categorizing (flexible), and conceptualizing (main tool)
What are the 4 aspects of Qualitative Data Analysis?
Collecting Data (interview, observations, documents) Thinking and Coding Data (transcripts, documents, fieldnotes) Noticing and Observing Discovering Patterns (categories -> concepts)
What makes up the Conceptual Framework Theory Development?
Comprehending (multiples sources of evidence/data, open and axial coding)
Synthesizing (Data reduction, narrative summaries, category and case records)
Theorizing (Patterns, test assertions, build explanations, consultation, disseminate preliminary findings)
Reconceptualizing (case reports, cross case analysis, emerging framework and theory)
What makes up the Standards of Verification?
Prolonged engagement and persistent observation Triangulation Peer review or debriefing Negative case analysis Clarifying researcher bias Member checks Rick, thick, description External audits (paper trail)
What are the typical errors in qualitative research?
Listing all comments without doing any analysis
Include info that doesn’t identify the respondent
Generalizing from comments to the whole group (Qualitative provides unique insights, can’t be generalized)
What are the 5 steps of Content analysis?
Get to know your data
Focus the analysis (what do you want it to answer)
Categorize the information (identify themes, organize them into coherent categories)
Identify patterns and connections within and between categories (sort by theme, larger categories, number of times those themes come up, show relationships among categories)
Interpretation - bring it all together
What is epidemiology?
Study of disease (or condition) distribution in populations
Includes factors that influence or determine the distribution
What is the purpose of epidemiology?
Identify etiology and risk factors
Determine impact on population
Study natural history and prognosis
Evaluate preventative and therapeutic measures
Provide foundation for developing public policy
Prospective vs Retrospective
Prospective:
Happening at this current time
Measure baseline to determine who becomes injured
Retrospective:
Determine who has the injury
Looking back at a group and analyzing them
What is a Prospective Cohort Study?
Gold standard for injury risk factors
Baseline testing on a number of factors
Followed over time to see if a condition develops (can take a long time)
Comparison of individuals who develop the condition versus individuals who don’t
Examples: Framingham (5,000+, Cardio and OA); Johnson County, NC (3000+, hip and knee OA)
What is a cohort?
Group of individuals who are followed together over time Common types: Geographic Generation Veterans of wars Natural Disaster Survivors
What is a Retrospective Case-Control Study?
Statistical analysis of measures between participants with condition (Cases) and participants without the condition (Control)
Examples: OA, obesity, joint injury, quads weakness, occupation
Can risk factors be determined in Case-Control studies?
No, it is not known if the outcomes are a cause or result of the condition
What are the pros of a Case-Control study?
Can provide valuable information for potential risk factors
Very useful if injury or illness is rare
Doesn’t require as many subjects -> less time
Many prospective studies start out as retrospective studies
What is an endemic?
Habitual presence of disease within a geographical area
What is an epidemic?
Occurrence in a community, beyond normal expectancy, derived from common source
What is a pandemic?
Worldwide epidemic
What is incidence?
Number of new cases in a given period of time (per hours, Athlete exposures)
What is prevalence?
Number of individuals with the given condition
Expressed as a percentage
Pre-test probability
Intrinsic vs Extrinsic Risk Factors
Intrinsic: inherent to the organism
Examples: gender, previous injury, skeletal (mal)alignment, musculotendinous (in)flexibility, Proprioception/Neuromuscular control
Extrinsic: outside the organism
Examples: Environmental surface (concrete, grass), protective equipment, rule change/occupational guidelines, human factors (aggression from opposition, accidents involving others)
What is Relative Risk?
Association between exposure to a factor and development of a disease
Proportion of injury incidence between groups (prospective)
Ratio of risk in the exposed group to risk in the unexposed group (control)
RR =
Exposed/Unexposed
What do Relative Risk scores mean?
RR = 1: risk is equal in exposed and unexposed
RR > 1: risk in exposed is greater than risk in unexposed (positive association)
RR < 1: risk in exposed is less than risk in unexposed (negative association)
Can Relative risk be calculated in case-control studies?
No, you don’t know the incidence of the exposed population, and all individuals in the exposed group have the condition
What is an odds ratio?
Ratio of the odds of having the condition in the experimental group relative to the odds of having the target disorder in the control group (not impacted by prevalence)
OR =
EOC/EOE
What do odds ratio scores mean?
OR = 1: event is equally likely in both groups
OR > 1: event is more likely in the EOC group
OR < 1: event is less likely in EOC group
Relative Risk vs Odds Ratio
Relative Risk: Relevant for cohort study (prospective) Can calculate incidence Can't be calculated from case-control study (no measure of incidence) Helpful if condition is common
Odds Ratio:
Can be used in case-control and prospective study
Helpful if disease is rare, but not helpful if common
Morbidity vs Mortality
Morbidity:
Number of persons in a population who become ill (incidence) or already have that condition (prevalence)
Mortality:
Total deaths/population at midyear
Why is survival analysis important?
Regression models (proportional hazards regression) Accounts for covariates (age, sex, occupation, comorbidities) Can be used to calculate odds ratio
Efficacy vs Effectiveness
Continuum, not a dichotomy
Efficacy:
Smaller study, more control
Often RCT with control group
Effectiveness:
Bigger study, less controlled
Examines outcomes in real life
Often RCT with comparison group receiving standard of care
What is efficiency?
Cost-benefit ratio of effective intervention
What is a Clinical Prediction Guide?
Combination of clinical findings (cluster) that provide meaningful predictions about an outcome (diagnosis/treatment) of interest
Systematically derived
Algorithm based
What are the 3 types of Clinical Prediction Guides?
Diagnostic
Prognostic
Intervention
What are the pros of Clinical Prediction guides?
Potential to improve outcomes Increase patient satisfaction Decrease costs of care in healthcare Establish diagnosis Prognostic value Enable effective management
What 3 things do you need to know when applying Clinical Prediction Guidelines?
Who the rule is intended
Performance characteristics
Conditions of when to apply
What statistical aspects do Clinical Prediction Guides provide?
Regression (relationship)
ROC
Shifts in post-test probability (+/- LRs)
What are the 3 steps in Clinical Predictor Guideline Development?
Step 1 Derivation (identify factors with predictive value): Level IV, figure out what factors need to be included
Step 2 Validation (reproduction of the rule in various population): Level III and II (narrow and broad), expand to other/similar populations and expand
Step 3 Impact Analysis (evidence that the CPG changes clinician behavior and impacts patient-centered outcomes and/or reduces costs): Level I, did it actually help
What CPGs exist in the PT world currently?
Manipulation and Lumbopelvic Manipulation
Everything else is still at Level I
Why are Clinical Prediction Guides helpful?
Can enhance accuracy and efficacy of decision making : algorithm based (pattern recognition), not a cookbook approach
Most helpful in clinical situation with complexity or uncertainty: guides diagnosis and treatment
May improve research by denoting homogenous subgroups for treatment outcome studies
What are the 4 factors in the Goldman Algorithm?
ECG
Unstable Pain
Fluid in the Lungs
Blood Pressure < 100
What are the 3 factors of the Canadian Cervical Spine Rule?
Three High-Risk Criteria
Five Low-Risk Criteria
Ability to Rotate Neck 45*
Clinical Prediction Guide vs Clinical Practice Guidelines
Clinical Prediction Guide:
Single investigation
Combination of factors to predict outcome (diagnosis, treatment): more focused
Clinical Practice Guidelines:
Systematic summary of findings
Comprehensive
What are Clinical Practice guidelines?
Recommendations to optimize patient care
Informed by a systematic review of evidence
Expert panel with clinician-scholars that are updated periodically
What is a Systematic Review?
Focused on specific question
Scientific methods to identify, select, assess, and summarize findings
Comprehensive analysis of the current clinical literature (secondary analysis)
Critical analysis, but bias may still exist
Summarizes qualitative and quantitative (meta-analysis) data
What is a Meta-Analysis?
Type of systematic review
Combines the effect from multiple studies to give an estimation of the overall mean effect
Data is pooled
Quantitative estimation of the magnitude of the effect (point estimate) and uncertainty (confidence limits)
What is a Narrative (Literature) Review?
Broad
May not answer clinical question (search usually not criterion-based)
Article appraisal may vary
Often qualitative summary (can be biased/take author’s position)
Components of Meta-Analysis and Systematic Review
Must have strict inclusion/exclusion criteria (design of study, minimum level of quality)
Individual study characteristics must be similar (design, population, intervention, outcome measure, timing of follow-up)
What are 2 guidelines that can be used for Meta-Analyses?
PRISMA: Preferred Reporting Items of Systematic Reviews and Meta-Analyses
MOOSE: Meta-Analysis of Observational Studies in Epidemiology
What is data synthesis and what are the assumptions?
Combine data from multiple studies Assumptions (homogeneity): Participants Study Design Outcomes Follow-up
What statistics are required in a Meta-Analyses?
Mean differences (consider publication bias) Findings are weighted based on sample size, not just a sample average
What contributes to the weighted average?
Sample size: larger sample = greater weighing
Variance: uncertainty (standard deviations), lower variance = more certainty = greater weighing
How are results presented in Meta-Analyses?
Statistically (include CIs) LRs ORs Relative Risk (RR, RRR, ARR, NTT) Effect size
What is Relative Risk Reduction?
Percentage that the treatment reduces risk compared to the control
RRR =
(1 - RR) * 100
Goal is 100%
What is Absolute Risk Reduction?
Absolute arithmetic difference in event rates between control and experimental groups
Decrease in risk of treatment in relation to a control treatment
ARR =
CER - EER
What is the Number Needed to Treat?
Measure used to evaluate the effectiveness of a healthcare intervention
Number of individuals that need to be treated to prevent one injury compared to the control
NNT =
1/ARR
Inverse of ARR
What is NNTB?
Beneficial/Preventative effect occurred due to intervention (1 is ideal)
What is NNTH?
Harmful effect occurred due to intevention
How do you present your results?
Forest Plot: treatment effects of individual studies, pooled treatment effect
Funnel plot to check for publication bias (lack of studies = publication bias)
What makes up the Discussion section of the Meta-Analysis?
Summarize main study findings (context with other evidence, include strength of evidence)
Discuss limitations of current literature
Future research
What tools are used for the strength of evidence?
Centre for Evidence-based Medicine (CEBM)
Strength of Recommendation Taxonomy (SORT)
Grading of Recommendations Assessment, Development, and Evaluation (GRADE)
What makes up the Systematic Review Validity?
Review limited to high quality studies
Detailed method sections
Publication bias is addressed
Meta-analysis: individual patient data, aggregate data
What is a scholarly clinician?
Clinician who appropriately uses steps of EBP to better inform clinical practice
What are the steps in practicing EBM?
Define clinically relevant question (ask)
Search for the best evidence (acquire)
Critically appraise the evidence (appraise)
Apply the evident
Evaluate the performance of EBM (assess)
What is a Clinician-Scientist?
Formal training in both clinical practice and rehabilitation research, are uniquely trained to ask and answer the important clinical questions that drive our practice forward
What is Case-Based Learning?
Good teaching tool
Student Facilitated (may be guided by senior/expert clinician)
Find best available evidence
What are the components of a Case report?
Examination Evaluation Diagnosis Intervention Outcome Assessment (Review of Literature) (Reflection)
What are Grand Rounds?
Often performed as a lecture/presentationValue derived from junior leaders (students, residents, novice clinicians)
Critical review of literatures with debate (usually uncovers more questions)
Professionalism (respect and facilitate student directed learning)
Why is clinical reasoning crucial?
Accountability may suffer when clinicians follow examination and treatment routines without considering or exploring alternatives
What are common characteristics of expert clinicians?
Knowledge (series of mentors, patient-derived, breadth and depth) Clinical Reasoning Movement (visual and hands on) Virtues (patient's best interests) Highly motivated (internal motivation)
What are important Clinical Reasoning Skills to possess?
Perception of relevant versus irrelevant information
Interpretation of information and hypothesis generation
Inquiry strategies
Weighting and synthesis of information
What is the heart and soul of EBP? Why is it so important?
Journal Club
Participants develop skills to be a connoisseur of literature (reflection, critical analysis, translate into practice)
Analysis of articles is similar to job of journal reviewer
Which of the following is not a purpose of case reports?
a. Build problem solving skills
b. Determine cause and effect
c. Develop hypotheses for research
d. Share clinical experiences
B. Determine cause and effect
A study found that the most common injury in baseball players are labral tears. The results yielded an ICC value of .88. Which of the following is a true interpretation of the results?
a. This study has excellent reliability.
b. This study has poor reliability.
c. This study showed that 88% of its participants had a labral tear.
d. This study showed that there was a ceiling effect of 88%.
A. The study has excellent reliability
The purpose of this prospective case series was to determine the MCID in Functional gate assessments (FGA) for the older community-dwelling adults relative to patients’ and physical therapists’ estimates of change and the extent of agreement between patients’ and physical therapists’ estimates of change. After completing this case series, the kappa statistics score was 0.163 (weighted kappa = 0.163). The MCID was 4 points, (SN = 0.66, SP = 0.84, LR+ = 4.07, LR- = 0.40). Interpret these findings.
a. A patient whose FGA score improved by 7 points would be considered to have a clinically significant change.
b. Prospective case series has higher level of evidence on the clinical research design compared to a cohort study
c. The kappa value in this series shows a moderate level of agreement
d. The LR+ shows a strong shift in probability of the patient having the disease or complaint
A. A patient whose FGA score improved by 7 points would be considered to have a clinically significant change
An elementary school health fair takes the weight of a child at the fair three times, on the same scale each time, within the hour he/she spends at the fair and gives an intraclass correlation coefficient of 0.76. This intraclass correlation coefficient indicates which of the following?
a. At minimum, 76 children needed to participate in this health fair to show any clinical meaning.
b. Not able to interpret the meaning of this value because a control group was not used.
c. The reliability of this scale measurement is good.
d. The weight of the child increased by 0.76 pounds during the three measurements.
C. The reliability of this scale measurement is good
Study A is evaluating the correlation between drinking an energy drink or sugar water with their corresponding blood pressure. Group I will drink one 16 fl oz can of RockStar energy drink per day. Group II will drink one 16 fl oz glass of water with 3g of sugar per day. Group III will be the control group and will drink one 16 fl oz glass of water per day. Blood pressure will be measured using a blood pressure cuff and stethoscope by a licensed physician at 5pm every day for five days. A volunteer sample of 25 individuals from the University of Minnesota Biology department is randomly assigned to one of the three groups. The average age of participants is 24.4 with an average pre-screen blood pressure reading of 120/80. How would you describe the validity of this study?
a. The blood pressure reading measurement does not have face validity because the measurement tools do not measure blood pressure.
b. The strength of the internal validity is increased because the researchers use independent variables to measure a difference in a dependent variable.
c. This study has strong external validity because it uses a convenience sample, which allows the results to be generalized to the real world.
d. This study is not internally valid because the study design does not allow the results to be generalizable to the real world
B. The strength of the internal validity is increased because the researchers use independent variables to measure a difference in a dependent variable.
The following contingency table is provided in the results section of Study A:
Labrum Tear Present Labrum Tear Absent
Labrum Test Positive A = 10 (true positive) B = 30 (false positive)
Labrum Test Negative C = 20 (false negative) D = 15 (true negative)
The researchers state that the sensitivity for this Labrum test is 0.33. Is the sensitivity value correct?
a. No, sensitivity does not have to do with contingency tables.
b. No, because the sensitivity equation is A/(A+B), which equals 0.25.
c. Yes, because sensitivity is D/(B+D), which equals 0.33.
d. Yes, because the equation for sensitivity is A/(A+C), which equals 0.33.
D. Yes, because the equation for sensitivity is A/(A+C), which equals 0.33
Study Corp is researching anterior knee pain in runners. Study Corp used the Visual Analog Scale during their research and wanted to analyze the results. The participant responses are as follows: · Mean 1 = 3.7 · Mean 2 = 6.1 · Pooled SD = 4.2 Using the equation d = (Mean 2 – Mean 1)/Pooled SD, the effect size of Study Corp’s research study would be considered: a. Moderate b. Small c. Strong d. Weak
A. Moderate
You create a new test to measure BMI and want to see how your calculated values compare to the values calculated by the Bod Pod. What type of validity are you referring to?
a. Concurrent
b. Content
c. Face
d. Predictive
A. Concurrent
A Timed Up and Go (TUG) test was conducted on 2,985 patients in various national Parkinson foundation centers across the country. An initial TUG test was performed and measured, followed by two weeks of physical therapy, with a second test being conducted with a MCID of 1.8 seconds. This MCID can be interpreted as indicating what?
- A high intrarater reliability among the two tests, helping improve the validity of the study.
- The agreed upon range of measurement error allowed between two tests.
- The minimum amount of change in a patient’s score that ensures the change was not the result of measurement error.
- What the physical therapist or patient would consider as a smallest amount of change needed to indicate improvement.
- What the physical therapist or patient would consider as a smallest amount of change needed to indicate improvement.
In manual muscle testing, muscles are given a grade of 3 (Fair) if the patient can take the muscle/muscle group through a full range of motion against only the resistance of gravity. In MMT, reliability has been measured “…among examiners and in successive tests with the same examiner, the results should be within one half of a grade (or within a plus or minus of the base grade)).” In a study, reliability for this grade was reported as ICC= 0.43. What does this measure tell us about the reliability of this grade?
a. The reliability of the grade is poor.
b. The reliability of the grade is fair.
c. The reliability of the grade is moderate.
d. The reliability of the grade is good.
a. The reliability of the grade is poor.
A 13 year-old patient comes in with ankle pain. The physical therapist utilizes the Ottawa Ankle Rule guidelines to determine whether the patient has a fracture. The test comes out negative. Knowing the Ottawa Ankle Rules has a high specificity but a low sensitivity, the PT can…
a. Can confidently rule in but cannot rule out the condition
b. Cannot rule in but can confidently rule out the condition
c. Cannot rule in and cannot rule out the condition
d. Unable to determine given the results
c. Cannot rule in and cannot rule out the condition
There are several checklists that are used in clinical trials to help researchers follow guidelines and help others critically appraise research. Which checklist allows quantification of the quality of a research design?
a. CONSORT
b. PEDro
c. PRIMSA
d. STARD
B. PEDro
A study examined the five time sit to stand test (5STS) as a functional outcome measure for patients with COPD. The results showed that the test-retest and inter-observer ICCs were 0.97 and 0.99, respectively. The 5STS scores correlated significantly with other measures of function or impairment such as the ISW, QMVC, SGRQ, ADO and iBODE (r=−0.59, −0.38, 0.35, 0.42 and 0.46, respectively; all p<0.001). The MCID for the 5STS was determined to be 1.7 s in this population. Which of the following is a correct interpretation of these results?
a. For each measure correlated to the 5STS, the risk of a Type I error is less than 1%
b. The smallest detectable change in measurements (not due to error) for the 5STS is 1.7s
c. The 5STS scores are more strongly correlated with scores of the SGRQ than the ISW
d. This test is considered to have good validity because the ICC scores are above 0.75
a. For each measure correlated to the 5STS, the risk of a Type I error is less than 1%
A treatment effect is found to have moderate strength. What is a possible effect size?
a. 0.27
b. 0.44
c. 0.52
d. 0.86
C. 0.52
A study examined vegetarians and their frequency of ACL tears over two years. One of the participants had a +LR of 3.57. A +LR of 3.57 indicated which of the following?
a. Strong probability of an ACL tear.
b. Moderated probability of an ACL tear.
c. Small probability of an ACL tear.
d. Very small probability of ACL tear.
c. Small probability of an ACL tear.
A new study looked at the effectiveness of aquatic therapy in treatment of knee and hip osteoarthritis. In the study, the researchers determined an AAR=0.50 and an NNT=2 for the reduction of pain in the aquatic therapy group. Based on this data, one could expect:
a. that 50% of the population in this study falls between the corresponding range of scores
b. to reduce knee pain in 1 out of 2 patients who participate in the aquatic therapy program
c. the absolute risk of knee pain was reduced by 20% for those that participated in the aquatic therapy session
d. the patient to improve their knee pain after completing two sessions of aquatic therapy
B. To reduce knee pain in 1 out of 2 patients who participate in the aquatic therapy program
A 38-year-old male comes into your clinic following a cervical injury surgery, who is looking to return to activity. Through the interview portion of your assessment, you discover he is a professional football player for the Indianapolis Colts. During your test and measures portion you observe the client achieving the highest measurements possible and at times exceeding the instruments available range. What phenomenon do you note? A. Basement effect B. Ceiling effect C. Floor effect D. Positive likelihood ratio
B. Ceiling effect
A physical therapist measured the ROM for a patient who was treated for a knee injury. The reliability had an ICC of 0.67. What does this reliability value show of this particular measurement?
a. Estimates of reliability for measures of continuous data are not reported as ICCs.
b. Good chance of reproducibility.
c. Moderate chance of reproducibility.
d. Poor chance of reproducibility.
c. Moderate chance of reproducibility.
A baseball player comes in to your clinic and presents signs and symptoms of a rotator cuff tear. You decide to use a disablement framework as a basis for this case and you chose the ICF model. Which of the following would be a possible participation limitation?
a. Lack of a support group at home
b. MRI confirmed tear in the right supraspinatus tendon
c. Pitching in a baseball game
d. Reaching for an item in an overhead cabinet
c. Pitching in a baseball game
Following the Center for Evidence-Based Medicine (CEBM). What study design is considered to be level 3 evidence?
a. Case series
b. Case control studies
c. Expert opinion and disease-oriented evidence
d. Prospective cohort studies
b. Case control studies
During a study conducted upon collegiate soccer players, the positive likelihood ratio of a new screening technique for medial menisci tears was determined to be 11.3. The negative likelihood ratio is 0.08. What is the most accurate statement concerning this new technique?
e. It is highly indicative that the target condition exists when a positive test occurs.
f. It is a poor indicator that the target condition exists when a positive test occurs, due to its low negative likelihood ratio.
g. It is a poor measurement because when the patient tests negative for the condition, it is often an erroneous result and the person has been misinformed.
h. It is difficult to say because the details of the incidence of the condition were not disclosed.
e. It is highly indicative that the target condition exists when a positive test occurs.
A soccer player has received a left hamstring strain during practice for the second time this year. Due to the injury he will not be playing in the team’s championship game in two weeks. This is an example of which of the following:
a. Injury Rate
b. Injury Risk
c. Prevalence of injury
d. Time loss injury
d. Time loss injury
A Receiver Operating Characteristic (ROC) Curve examines the tradeoff between sensitivity and specificity when you select different cutoff values for a specific test. If the area under the ROC curve has an area of 1.00, what can be inferred from the test?
a. The test has one false positive and zero false negatives
b. The test has zero false positives and zero false negatives
c. The test is classified as good since the area under the curve is 1.00
d. The test is no better at identifying true positives than flipping a coin
b. The test has zero false positives and zero false negatives
A physical therapist wants to test a patients Lumbar Spine ROM and pain. He narrows down his test of measurement to two movement; sidebending or rotation. Both movement can test the ROM and pain of the lumbar spine. He knows that sidebending has a kappa statistic of .60 and rotation has a kappa statistic of 0.17. These kappa statistics indicate which of the following?
a. Sidebending has a higher percent agreement than rotation.
b. The kappa statistic of sidebending is substantial and the kappa statistic of rotation is fair.
c. There is a 60% chance that sidebending will be a type 1 error.
d. There is a 17% disagreement for rotation and 60 % disagreement for sidebending to check lumbar spine ROM and pain.
a. Sidebending has a higher percent agreement than rotation.
You have conducted a study looking at the effects of a new manual therapy technique on shoulder pain. After collecting and analyzing your data, you get a p-value of 0.25 (p=0.25). Which of the following is the most likely explanation for your finding?
a. You committed a Type I error, likely because your effect size was too small
b. You committed a Type I error, likely because your variance was too large
c. You committed a Type II error, likely because your sample size was too small
d. You committed a Type II error, likely because your statistical power was too large
c. You committed a Type II error, likely because your sample size was too small
A study comparing the flexibility of older adults who stretch versus those who do not was conducted. The researchers reported a p value of 0.03. Which of the following is true regarding this p value?
A. The researchers must fail to reject their null hypothesis
B. The results are clinically significant
C. The results are neither clinically nor statistically significant
D. The results are statistically significant
D. The results are statistically significant
In The five-repetition sit-to-stand test as a functional outcome measure in COPD article, the same 50 patients were measured simultaneously for the 5 time sit-to-stand test by two observers on the same occasion. What type of reliability does this demonstrate?
a. Concurrent
b. Content
c. Intrarater
d. Interrater
d. Interrater
If a researcher is designing a study and is trying to make sure their measurements have concurrent validity, what should the researcher be looking for?
e. How well their measures represent the constructs they are supposed to measure.
f. How well the measures correlate with the existing gold standard measure.
g. The ability of different testers to repeatedly produce consistent measurements.
h. The Property of whether their specific measures assess what they are designed to measure.
f. How well the measures correlate with the existing gold standard measure.
A Therapist is seeing a patient for knee pain and wants to rule out whether or not the patient’s pain is caused from an ACL tear. The Therapist decides to use the Drawer Test and obtains a positive test, along with the following data: +LR = 11, -LR = 0.3. Based off this information, what can the therapist safely conclude?
a. The therapist can definitively rule in the possibility that the patient has an ACL tear.
b. The therapist can definitively rule out the possibility that the patient has an ACL tear.
c. The test revealed a large likelihood that the patient has an ACL tear.
d. The test revealed a small likelihood that the patient has an ACL tear.
c. The test revealed a large likelihood that the patient has an ACL tear.
A given special test used for identifying a torn rotator cuff has a specificity of 0.93 and a sensitivity of 0.02. A physical therapist performs this special test on a patient and finds the results to be positive. What does this test result reveal to the therapist?
a. Can confidently rule in a torn rotator cuff
b. Can confidently rule out a torn rotator cuff
c. There is a 2% chance of a torn rotator cuff
d. There is a 93% chance of a torn rotator cuff
a. Can confidently rule in a torn rotator cuff
The gold-standard of radiography is used to establish validity of goniometric measurements. Which type of validity correlates with this gold standard?
a. Concurrent Validity
b. Construct Validity
c. Content Validity
d. Face Validity
A. Concurrent Validity
You are assessing a patient for pain in their lower back. At the initial evaluation, the patient stated that their pain was an 8.5/10. What value is appropriate to conclude that there is a significant improvement or deterioration in the status of the patient’s pain?
a. The patient is sedated and unable to give an accurate rating
b. The patient reports their pain at a 7.5/10
c. The physical therapist makes a conclusion that the patient is better without using the scale
d. The rated pain from the patient is 5/10
d. The rated pain from the patient is 5/10
A measure that addresses all aspects of the object or item being measured is represented by which type of validity?
a. Concurrent validity
b. Construct validity
c. Content validity
d. Convergent validity
c. Content validity
A physical therapist found a change of 5 degrees of ROM in the elbow after treating a patient for 4 weeks and wants to know if she can document this as clinically meaningful based on minimal clinical important difference (MCID). What information will the physical therapist receive from the literature on MCID related to elbow ROM?
a. Degree of elbow ROM improvement that is meaningful to patients
b. MCID will not give the therapist any information relevant to elbow ROM
c. The p-value is < 0.05 and therefore the ROM is clinically meaningful
d. The therapist can say that 5 degrees of ROM is clinically and statistically significant
a. Degree of elbow ROM improvement that is meaningful to patients
A study found that low intensity training with blood flow restriction can improve peripheral blood circulation in elderly adults. Data looking at muscle strength levels for a one rep max leg press indicate a difference between the blood flow restriction group and the non-blood flow restriction group of (F = 11.7, P < 0.01). How would you interpret the P value?
a. Don’t reject the null hypothesis because the result is due to chance
b. Since P < 0.01 there is little if any correlations between the groups
c. There is a statistically significant difference between groups
d. There is not a statistically significant difference between groups
c. There is a statistically significant difference between groups
As a physical therapist and researcher, you are developing a study that examines a new special test for ACL tears. You decide to test for the correlation between the special test results and MRI results. By doing this, you are adhering to what type of validity?
a. Concurrent
b. Content
c. External
d. Internal
a. Concurrent
The Berg Balance Scale has a SEM of 2.3 for institutionalized older adults. If an institutionalized older adult improved their Berg Balance Scale score from 28 to 35 points, is that considered to be due to error and what is the MDC?
MDC = 1.96 x SEM x (square root of 2)
a. MDC = 9.02, Yes the increase in score could be from measurement error
b. MDC = 9.02, No the increase is not from error and is true improvement
c. MDC = 6.3, Yes the increase in score could be from measurement error
d. MDC = 6.3, No the increase is not from error and is true improvement
d. MDC = 6.3, No the increase is not from error and is true improvement
Jones S, Kon S, Canavan J, et al. determined the MCID for the five-repetition sit-to-stand (5STS) was 1.7 seconds. How would this be interpreted for clinical application?
a. If your patient improved by 1.7 seconds it is clinically significant.
b. If your patient’s score was 1.7 seconds slower than norm values they were at risk of falling
c. You can expect to have aa 1.7 second difference between testers.
d. Your patient’s score must be 1.7 seconds faster than norm values to be clinically significant.
a. If your patient improved by 1.7 seconds it is clinically significant.
A patient comes in to see you at the clinic. The patient is a 78-year old male with Parkinson’s disease. His wife has come in because she has been worried about him falling and concerned about his walking ability. The following are test and measures that are specific to the Parkinson’s population:
- Five-time sit-to-stand (test/retest reliability ICC= 0.76, inter-/intrarater reliability ICC=0.99, sensitivity= 0.89, specificity=0.47, cut-off score for risk of fall is >16 seconds)3,5,6
- Four Step Square Test (test/retest reliability ICC= 0.78, inter-/intrarater reliability ICC=0.99, sensitivity= 0.73, specificity=0.57, cut-off score for risk of fall is 9.68 seconds.)2
- Functional Reach Test (test/retest reliability ICC= 0.84, inter-/intrarater reliability ICC=0.74-0.64, sensitivity= 0.52, specificity=0.53, cut-off score for risk of fall is >16 seconds)1,10
- Timed Up and Go Test (test/retest reliability ICC= 0.85, inter-/intrarater reliability ICC=0.99, sensitivity= 0.69, specificity=0.62, cut-off score for risk of fall is <25.4 cm.)4,8,9
Which test is the safest and most appropriate test to administer to test the patients fall risk and gait?
a. Five-time sit-to-stand
b. Four Step Square Test
c. Functional Reach Test
d. Timed Up and Go Test
d. Timed Up and Go Test
You are reading an article about an imaginary test and find that this particular test has an MCID of 10 and a MDC of 5. If your patient were to score an 8 on this test what could you conclude from this information?
a. The patient has had a change that is due to measurement error and this change is clinically significant.
b. The patient has had a change that is due to measurement error and this change is not clinically significant.
c. The patient has had a change that is not due to measurement error and this change is clinically significant.
d. The patient has had a change that is not due to measurement error and this change is not clinically significant.
d. The patient has had a change that is not due to measurement error and this change is not clinically significant.
A patient comes to you for ankle pain, so you perform a test to assess if he has torn his calcaneofibular ligament. This test has a specificity of .89, a sensitivity of .37, a positive likelihood ratio (+LR) of 11, and a negative likelihood ratio (-LR) of 0.4. If your test got a positive result, how would you interpret these findings?
i. Rule out the condition based on the specificity and +LR
j. Rule out the condition based on the sensitivity and -LR
k. Rule in the condition based on the specificity and +LR
l. Rule in the condition based on the sensitivity and -LR
k. Rule in the condition based on the specificity and +LR
What is a positive cardiovascular disease (CVD) risk factor?
q. Body Mass Index (BMI) ≥ 30 kg/m2
r. High Density Lipoprotein (HDL) ≥ 60 mg/dL
s. Fasting plasma glucose ≤ 126 mg/dL
t. Systolic blood pressure ≤ 130 mmHg
q. Body Mass Index (BMI) ≥ 30 kg/m2
Diagnosis of a superior labral tear from anterior to posterior (SLAP tear) on conventional MR images has a reported sensitivity of 98%. These values are representative of which of the following?
a. In a negative test, a SLAP tear can be ruled in by using MR imaging
b. In a negative test, a SLAP tear can be ruled out by using MR imaging
c. In a positive test, a SLAP tear can be ruled in by using MR imaging
d. In a positive test, a SLAP tear can be ruled out by using MR imaging
b. In a negative test, a SLAP tear can be ruled out by using MR imaging
MCID for VAS
1.5 - 2.0 cm
MCID for NPRS
2 points
MCID for LEFS
9 points
MCID for FAAM
ADL: 8%
Sport: 9%
