Comps review Flashcards
When/why ask an FQ?
Uncertain about clinical issue, want an answer
What is a PICO?
4 required elements of FQ (in any order)
What does PICO stand for?
- Patient/problem
- Intervention
- Comparison/contrast
- Outcome
What to use PICO for?
- Research about treatment
- about diagnoses/screening tools
- How well one of the treatments/diagnosis tools worked for a client
- How you would gather patient preferences about their treatment options
Oxford Hierarchy (top to bottom)
- Systematic review and meta analyses of RCTs
- RCTs
- Cohort studies
- Case control studies
- Cross sectional surveys
- Case studies
- Ideas, expert opinions, editorials
- Anecdotal
Lit reviews
- Systematic review
- Meta-analysis
Systematic review
Gather and summarize all relevant studies on a topicM
Meta-analysis
If the studies have similar enough methods, pool them and do stats over everything
- Numerical support for the conclusions across studies
Individuals works
- Lit reviews
- Original research
Professional association journals and websites
- ASHA
- American Academy of Audiology
- American Psychological Association
Documenting steps
- Heading: Where you searched + search terms
- List full citations for articles that look relevant
- List notes below the citation about the article’s usefulness for your current purpose
Citation parts
- Authors
- Year
- Title (article, chapter)
- Source (journal, book)
- Publication details
- Page numbers
- DOI or website
Clinical Studies: Phase Model
- Phase I & II: Exploratory, small groups
- Phase III: Hypothesis testing, big samples
- Phase IV: Translate to practice
- Phase V: Practical matters
Phase I & II: Exploratory, small groups
- Treatment effect
- Refine operations, populations, methods, effects
Phase III: Hypothesis testing, big samples
- Treatment efficacy
- Pretest-posttest
Phase V: Practical matters
- Cost-benefit
- Quality of life
- Satisfaction
Review Articles
- Summarize results from Phase IV & V studies w/ common hypotheses
Review Styles
- Narrative
- Meta-analysis (quantitative)
- Best evidence
Narrative Review - Traditional lit review
- Thorough search
- Describe results qualitatively
- Overall conclusion
Narrative Review - Drawbacks
- Subjective bias
- Subjective interpretations
Systematic Review
- Clear protocol for selecting and evaluating studies before beginning review
- Has 6 steps
Steps to Systematic Review
- Formulate problem/question
- Locate, select studies (selection criteria)
- Assess study quality (uniform standards)
- Collect data (across studies, quantitative or qualitative methods)
- Analyze results
- Interpret results
Early Meta-Analysis Methods
- Vote counting
- Combined-probability
Vote Counting
- Number of studies with positive, negative, null results/conclusions
- Drawback: no effect size
Combined-probability
- Incorporate probabilities (account for different sample sizes)
- But still no effect size
Modern Meta-Analysis Outcome
Overall effect size and significance across studies w/ similar quantitative methods
Modern Meta-Analysis
- Good way to combine results of studies on different populations, small samples, etc
- Strong evidence for clinical decisions
- Identify gaps, ideas for future research
Best Evidence Approach
- Combines “best” of narrative & meta-analysis
- Attempts to avoid drawbacks
Combines “best” of narrative and meta-analysis
- Narrative intro, discussion, conclusion
- Objectivity in selection criteria, evaluating quality
- May use quantitative/meta-analysis
Attempts to avoid drawbacks
- Bias in study selection
- Balance between big picture and important points from individual studies
Good review features
- Clear scope, purpose, theories
- Systematic, thorough evidence search
- Systematic appraisal of all studies for relevance, quality/rigor
- Sound synthesis across studies
- Reasonable conclusions based on synthesis
Reading Article Order
- Abstract: is this article relevant?
- Introduction: find the research question
- Find answers in conclusion
- Start from the top: get context from lit review
- Methods: evaluate study quality
- Results: evaluate rigor
Research Ethics
- Fair treatment of research participants
- Honesty, accuracy in reporting
Fair treatment of research participants
- Minimized harm, maximized benefit
- Informed consent
- Protected private data
Honesty, accuracy in reporting
- Describing procedures
- Minimizing subjective bias
- Giving credit
Participant Rights
- First, do no harm
- Nuremberg code
- Institutional review boards
- Belmont Reports
Nuremberg Code (1947)
- Voluntary consent: Free choice to participate
Institutional Review Boards (IRBs)
- Review research proposals BEFORE they begin
- Participants’ rights, protections; risks, benefits
Belmont Report (1979)
Codes for research with human subjects
- Medical
- Behavioral
Belmont Report
- Applies to human research participants
- Applies to research, not practice
- Respect for persons: informed consent
- Beneficence: risk-benefit assessment
- Justice: selection of participants
Respect: Informed Consent
- Informed of procedures, risks, alternatives
- Understand, make free choice to participate
- No coercion: Rewards can’t be too enticing
- Can quit any time and still get compensation
- Extra protections for vulnerable populations
Deception
- Only when the truth upfront would make the experiment impossible
- Must minimize risks of harm due to deception
- Must debrief at end
Beneficence: Risk-Benefit
- Ensure well-being of participants
- Do no harm
- Minimize risks, maximize benefits to participants
- Risk to participants doesn’t exceed benefit to science
Justice: Participant Selection
- Fair distribution of risks and benefits
- Minimize selection bias
- Subject should correspond to research purpose
Convenience Sampling
- Easy-access populations
- Prisoners
- Students
- Family members
Vulnerable populations
- Institutionalized
- Children
- Disabled
- Students
- Patients
- Immigrants
- Poor
Participants
- Anyone involved who’s not a researcher
Distributive Justice
- Participant pool should match the purpose of the study
- Inclusion/exclusion of participants based on need
- Purposefully exclude people who may benefit
- Purposefully include/select samples based on convenience or vulnerabilities
Other Issues
- Honoring commitments to participants
- Withholding treatment
- Conflicts of interest
- Privacy, confidentiality
- Data management, ownership, security
Honoring Commitments to Participants
- Compensation
- Continued therapy
- Summary of results
Withholding Treatment
- No treatment control groups: may feel unfair to “let people go untreated”
- Risk-benefit: is no-treatment harmful?
Conflicts of Interest
- When researcher has another role/interest related to the research/outcomes
- Teacher can’t recruit own current students
Privacy, confidentiality
- Identifying into = confidential unless stipulated in consent form
- Anonymize data: Use subject code w/ all data, store name-code key under lock and key
Data Management, Ownership, Security
- Follow collection, analysis protocols systematically
- Store securely
- Later: who “owns”
Reporting
- Honest, accurate description of study
- Responsibility to publish publishable results
Author Order
- First author = did most work
- Fields/labs differ on rest
- Most to least work
Validity
How closely something reflects reality
Internal Validity
Accuracy of relation between observations and the subjects observed
External Validity
Generalizability
Generalizability
Applicability of patterns/results to a larger population
Internal Validity Parts
- Confounders
- Subjective bias
Confounders
Unintended, uncontrolled, or unknown facts that should affect the results
- Alternate explanation
- Nullification
- False conclusion
Subjective Bias
Could influence any stage of research
Ways to minimize subjective bias
- Blinding
- Outside observer
- Reliability checks
Blinding
Make involved people unaware of information that could bias findings
Single-blind
Either patient or practitioner are unaware of the patient’s treatment group assignment
Double/triple-blind
- Other researchers are unaware of something
- Researchers who interact with subjects, give treatments, evaluate progress, analyze data
Norm-Referenced Tests
- Standardized
- Rank a score relative to normative sample
- Not designed for multiple administrations
Criterion-referenced
Compare performance to reaching an expected level
Consistency of Measurement
- Train examiners
- Monitor consistency of test admin procedures
- Check intra-examiner reliability
- Check inter-examiner reliability
Intra-examiner Reliability
Consistency of an examiner’s measurements across test subjects
Inter-examiner Reliability
- Consistency of measurements across examiners
- When examiners score same person/take same measurements, do they agree?
Randomized Controlled Trials (RCTs)
Best for causal inferences about average effects across a population
What are RCTs not appropriate for?
- Diagnostic accuracy
- Etiology
- Risk factors
- Rare/slowly-progressing conditions
- Risky/unethical experimental procedure
Experimental Design
Include active manipulations
Observational/Non-Experimental Design
- No active manipulations
- Observe systematically, don’t alter
Controlled Studies
Include control comparison group
Uncontrolled Studies
No control group
Controlled Trial
- One group receives treatment/manipulation, control group does not
Multiple Baseline Controlled Trial
Treatment group/patient is its own control:
- Measure multiple times before treatment, part-way through, after, later follow-up
Uncontrolled Trial
All participants receive treatment
- No control/comparison group
Cohort
Groups differing on a variable are followed over time to observe differences in outcomes
Case-control
Compare group with disorder to controls (w/o disorder), usually at one or a few points in time
Cross-sectional
Examine relationships between variables in a sample at one point in time
Case Study/Report
Describe single patient
Case Series
Describe series of similar patients
Prevalence/Surveillance Studies
Examine rates of occurrence in a sample
Prospective
- Hypothesis testing, methods planned out before data collection
- Experimental studies must be prospective
Retrospective
- Analyze pre-existing data
- Ranked lower than prospective: no control over systematic or unknown influences, can’t assess validity of procedures
Random Assignment
- All subjects have equal chance of being assigned to any condition, determined by chance
- Applies to prospective, controlled, experiments
- “Best way” to assure that groups don’t differ systematically before beginning
Matched Assignment
- Create groups that differ on the variable of interest but not others that are expected to influence results
- Weaker validity than random assignment: unknown, unanticipated confounders possible
Confounders
Unintended, uncontrolled, or unknown factors that could affect the results
Statistical Significance
Math that says whether or not your result was probably a fluke
P < .05
There is a 95% chance that the samples were not drawn from the same population
Generalizability
Applicability of patterns/results to a larger population
Subjective Bias
Minimize bias in participant selection
Random Assignment in Observational Studies
All members of the population have equal chance of being selected for observation
Attrition
- In studies with multiple measurement points, not all participants complete all steps
- Must report in published results
Replicability
Enough detail reported that another researcher could repeat the procedures? And get the same results
Continuous
Can have infinitely small, intermediate values
- Interval
- Ratio
Categorical/Discrete
Completely separate bins
- Nominal
- Ordinal
- Interval or Ratio data that has been binned into categories or ranges
Nominal - Unordered, named category labels
- Categories aren’t better/worse, higher/lower
- Demographics, type
- Best if every participant fits into just one category
Nominal - Can’t do most stats
- Even if assigning arbitrary numbers to categories
- Can count numbers of members
Ordinal
- Categories are ordered but there’s no “amount” of difference between levels
- Likert, rating, severity scales
- Hard to do stats on these, must transform
Interval
- Ordered with equal intervals: can compute differences between scores but not ratios
- Good for transformations and stats
Ratio
- Like interval plus true zero, can compute differences and ratios
- Great for transformations and stats
Frequency Distribution
How many data points fell in each interval
- AKA frequency polygon
Skewed
Long tail
- Positive/right skew = positive tail
- Negative/left skew = negative tail
If mean = mode
Skewness is 0
If mean > mode
Skewness is positive
If mean < mode
Skewness is negative
Bimodal
Two modes
- Mean misleading
- Really represents 2 distributions
Data Transformations - Modify raw values to simplify data structure
- Make distribution more symmetrical/normal
– Many stats require a normal distribution - Make validity more constant
- Make relationships more linear
- Convert ordinal data to interval/ratio scales
Data Transformations - Inspect distribution of data
- Looks normal? Outliers? Mistakes?
- Calculate skew, kurtosis to confirm
Nonlinear Transformations
Reduce relative spacing between values on the right more than left side of distribution
Square Root
Take square root of each value
- First add constant to make lowest value > 1
Log
Changes spread of distribution
Inverse: 1/x
- Makes big numbers small and small numbers big
- Reverses order of values, so first multiply each value by
– 1 and add a constant so the lowest value is > 1
Descriptive Stats
Summarize characteristics of data set
Counts
- Frequency
- Percentage
Location/Central Tendency
- Mean
- Median
- Mode
Individual Location
- Rank
- Percentile rank
- Standard score
Variability (spread)
- Range
- Variance
- Standard deviation
Frequencies
How many subjects/items in each category
Percentages, proportions
Divide each frequency count by total
Location/Central Tendency
- Single values that describe whole data set for one measure
- Central location/tendency
- Fractiles/Quantiles
Central Location/Tendency
- Mean (average)
- Median (middle value)
- Mode (most common value)
Fractiles/Quantiles
Divide rank-ordered date into even-ish bins
- Median split (2)
- Quartiles (4)
- Deciles (10)
- Percentiles (100)
Individual Location
Location of participant in relation to group
Rank
The Xth best score (out of?)
Percentile Rank
- Rank-order scores
– Divide individual’s rank by total number of participants - 80th percentile = scored better than 80% of class
Standard Score (z-score)
Number of standard deviations from the mean
- X - mean/st dev
Variability (Spread)
- How spread out the data values are
- Adds necessary meaning to central tendency and individual location
– Number of categories
– Range
– Interquartile Range
