Exam 1 Flashcards
Define EBM
- Use of mathematical estimates of risk of benefit or harm, derived from research on samples to inform decision-making in clinical setting of diagnosis, investigation or mgmt. of individual patients.
Steps of EBM
- Ask answerable question (PICO – see flashcard) - Search for best evidence - Critical appraisal for validity and relevance - Integrate evidence, clinical expertise and patient values/preferences & apply - Evaluate results
Elements of an answerable question
- PICO - P: patient/problem/population - I: intervention - C: comparison intervention - O: outcomes
Types of clinical questions
- Background: general knowledge, typically for students, who/what/when/where etc - Foreground: seek specific knowledge for patient management (comprise ¾ PICO elements), for clinicians
What is the best medicine?
- Patient-centered
Qualities of best evidence in medicine?
- Current, valid and clinically relevant
Hierarchy of research studies by type and reliability
- Systematic review of RCTs (or meta-analyses) 2. RCTs 3. Prospective studies (typically cohort studies) 4. Retrospective studies (typically case-control) 5. Cross-sectional surveys 6. Case series 7. Case reports
Initial steps to evaluate a research paper
- Ask: Why study was done? What type of study was it? Was the study design appropriate?
Structure of research papers
- Mnemonic = A(IMRAD) - Abstract: summary - Intro: why research done - Method: how research structured - Result: findings - Analysis - Discussion
When evaluating a research paper, it is important to ask why the study was done and what hypothesis was tested? Where in the paper can this information be found?
- Intro - Hypothesis usually in intro, if not, in methods. Rarely is it found in the discussion (first paragraph).
Primary vs secondary studies
- Primary: experiments, observations, clinical trials, surveys, questionnaires 2. Secondary: reviews (systematic or non), economic analyses, decision analyses
Types of study questions. What study design is appropriate for each type?
- Therapy/intervention: RCT - Diagnosis: diagnostic validation study - Prognosis/prediction: cohort study - Harm/risk/etiology: RCT, cohort study, case-control study - Screening: ?
Define randomized controlled trial
- Subjects randomly assigned to one intervention group or another by random method. Groups should be similar, on average, except for the outcome.
Purpose of randomization
- Decreases selection bias while at the same time creating similar comparison groups.
Define concealed allocation vs blinding
- Concealed allocation: during RCT, randomization of subjects done by someone other than the investigator. - Blinding: prevents investigator or subject (single) or both (double) from knowing the group assignment (tx vs non-tx group).
Purpose of blinding
- Help avoid patients’ behaviors and ideas about treatment affecting results - Help avoid investigators inadvertently altering or changing the study results
Pros/cons of a RCT
- Pro: rigorous eval of a single variable, designed prospectively (less bias), seeks to confirm a null hypothesis, allow for meta-anlayses, minimize bias - Con: expensive, long-term (years sometimes), hidden bias (inadequate randomization, failing to randomize all eligible patients – investigator only offers entry to those who may benefit, failing to blind – see what you want to/expect to see)
Which studies tend to have less bias: retrospective or prospective?
- Prospective. We tend to see what we are looking for through our retrospectors.
Problems with randomization
- Can be unethical. Never randomize in harm study. - Impractical when # of subjects needed for statistical significance is huge
When is randomization inappropriate?
- Study involves prognosis of a disease (unethical) - Validity of a diagnostic / screening test - Investigating quality of care issues when criteria for success are not known yet
Which study designs are observational?
- Cohort studies - Case-control studies
What is a cohort study?
- Two or more groups selected based on exposure or no exposure to something in order to compare outcomes
What is a case-control study? What are case reports?
- Patients with certain conditions are matched with controls usually retrospectively for exposure to a disease-causing agent or circumstance. These studies are mostly concerned with harm or etiology. - Case reports: report of a medical history, sometimes a series of histories are reviewed/analyzed together. Weak statistically usually, but may give insight into rarities or unusual things.
What is a cross-sectional survey?
- Representative sample is interviewed, examined or evaluated about a specific question. Often information is retrospective. Can be used in studies looking to answer questions about etiology.
Studies used for etiology questions
- Case-control and cross-sectional survey
What is a systematic review?
- This is an evaluation/review comparing RCTs, combination of data from RCTs.
T/F. Systematic reviews include all original reports (even those unpublished) available pertaining to the information in the paper.
- True
T/F. Systematic reviews make conclusions based on studies with no pre-set quality criteria.
- False. Conclusions based on studies with pre-set quality criteria
Advantages of systematic reviews
- Large amounts of info assimilated quickly - Explicit limitation of bias based on selection of studies - Studies compared for consistency and generalizability - Inconsistencies between studies easily identified - Conclusions are more reliable - Meta-analyses increase precision of results (don’t have to be present)
What are meta-analyses?
- Statistical synthesis of numerical results of several studies which all addressed the same questions. Incorporates advantages of systematic reviews with powerful statistical analyses. - In other words: every meta-analysis has a systematic review process, but not every systematic review will use meta-analysis statistical synthesis.
Cohort studies are typically retrospective or prospective?
- Prospective (cohort)
Case-control studies are typically retrospective or prospective?
- Retrospective (case-control)
Are diagnostic studies randomized?
- No
How are diagnostic validation studies designed?
- Take a population/sample - Run an experimental test on them, see if positive or negative - Use the reference standard/test on this same population, see if positive or negative - Look at sensitivity vs specificity of experimental test
Questions to ask when appraising diagnosis study?
- Gold standard used to confirm the presence of absence of disorder? - Was comparison between experimental test and gold standard blinded? - Was test evaluated on an appropriate spectrum of patients?
What are important factors to look for when appraising the results section of a diagnosis study?
- Sensitivity vs specificity. An ideal test will produce a high proportion of TPs and TNs - Pre- vs post-test probability - Likelihood ratio
Sensitivity
- How well does a test find people with the condition – TP/(TP+FN) - Mnemonic = SnNout = sensitive, negative, out. So, if test has high sensitivity and a negative result is found, then there is a strong chance to rule out that the person doesn’t have the condition. Why? It does a good job finding those with the condition.
Specificity
- How well does a test find people without the condition – TN/(TN+FP) - Mnemonic = SpPin = specificity, positive, in. So, if test has a high specificity and a positive result is found, then there is a strong chance to rule in that the person has the condition. Why? It does a good job finding those without the condition.
Pre and post-test probability. How can you tell if a diagnostic test is good based on these values?
- Pre-test: estimated probability of disease before the test result is known. This is based on prevalence in population, specific population etc. - Post-test: patient’s probability of having the dz after the test results is known. This tells you if X test really works to delineate the presence of disease. ** Note: a good diagnostic test increases the post-test probability significantly. When these are similar values or equal, then the diagnostic test is not very useful.
What is a likelihood ratio? What does a high LR indicate clinically? Low? Value of 1?
- Predict likelihood of certain result in a patient with the target disorder compared to the likelihood of the same result in one without. - LR >1: increases the probability, ie. the positive test is more likely to occur in people with the dz than in those without. Diagnostic test is helpful. - LR
Most important question to ask when critically evaluating diagnostic test studies
- Will you patients be better off as a result of the test? Does adding the test (and the information obtained from the test) change management that is ultimately beneficial?
What characteristics deem a diagnostic test valuable?
- Target disorder is harmful if undiagnosed - Test has acceptable risks - Effective tx exists
Most common kind of clinical papers
- Therapy studies
Are therapy studies randomized or non-randomized?
- Randomized.
How to assess for validity of therapy studies?
- Were all groups randomized? - Were all subjects accounted for? If too many lost, results can be skewed. Is loss comparable between groups? If less than 80% (lost > 20%) completed follow up, likely invalid study. - Were the participants blinded? - Was tx equal between groups? - Did randomization produce comparable groups at the beginning?
What is the most important thing to look for in RCTs?
- Concealed allocation
What is the ideal blinding?
- No one (researcher, subject) know who receives actual treatment, ie. double-blind.
What is intention to treat?
- All data from subjects in the intervention group is analyzed even if: they withdrew before study complete, they failed to take the assignment treatment, they were reassigned to a different group.
What happens if you fail to analyze subjects within their assigned group in a RCT?
- You can skew results toward intervention
Difference between accuracy and precision?
- Accuracy: did you hit the target? - Precision: did you hit the middle of the target?
What is called when results have been influenced in some way (each and every time) that lead to a false conclusion?
- Systematic error, a type of bias
How to determine the result from a therapy study is d/t chance alone?
- Use: o Use P value: P
What does a p value of
- P
What does a confidence interval that crosses 1.0 mean?
- The result of interest is not statistically significant.
What is EER, CER, ARR, RRR and NNT?
Control Experimental Event A B No Event C D - Control event rate (CER) = A / (A+C) - Experimental event rate (EER) = B / (B+D) - Relative risk (RR) = EER/CER - Absolute risk reduction (ARR) = CER – EER - Relative risk reduction (RRR) = ARR/CER - Number needed to treat (NNT) = 1/ARR
What is NNT (number needed to treat)? What is the clinical relavence of this?
- Number of patients you would have to treat to achieve one good result or to avoid one negative result. NNT = 1/ARR - Lower the NNT, the more useful the intervention to your practice
Hoff made a big deal about not confusing prognosis studies and harm/risk/etiology studies. Tell me about it….stud?
- Harm/Risk/Etiology: what happens of happened as the result of an exposure? - Prognosis: possible outcomes of a disease or condition and/or frequency with which they may occur.
What type of study designs are appropriate for harm/etiology questions?
- Cohort (prospective), case-control (retrospective)
What is the measure/estimate of risk in a harm/risk/etiology study that has a cohort study design?
- Relative risk
What is the measure/estimate of risk in a harm/risk/etiology study that has a case-control study design?
- Odds ratio
What does an OR or RR crossing 1.0 mean?
- Suggestive of no effect
What are prognostic factors?
- Characteristics of patient/population used to more accurately predict that patient’s/population’s outcome. Eg. = demographic, disease-specific, comorbidity
When you see natural history of an ailment in a paper, it is likely a what type of study?
- Prognosis study
What is the best study design for a prognosis study?
- Cohort - Case-control to determine prognostic factors
What is referral bias?
- This is a systematic error that can occurs when people who tend to participate (or are referred to a study) in a study will be different to controls. As a result, this increases the likelihood of adverse or unfavorable outcomes.
How do you evaluate the magnitude of results from a systematic review?
1.) Odds Ratio: describe the odds of a given event in a patient in tx group compared to the odds of that event in a patient in the control group. 2.) Relative Risk: describe the risk of a given event in a patient in tx group vs the risk of an event in a patient in the control group.
Odds ratio are suited for what studies? What does an odds ratio of 1 mean?
- Odds ratio describes the odds (likelihood) of a given event in a patient in tx group compared to the odds of that event in a patient in the control group. From Geletta: the odds that a person with the dz is exposed to a potential cause for the dz relative to the odds of a person without the disease is exposed to the potential cause. - Case-control - OR = 1 implies that the event is equally likely in both groups
Relative risk (risk ratio) are suited for what studies? What does an odds ratio of 1 mean?
- RR describes the risk of a given event in a patient in tx group vs the risk of an event in a patient in the control group. From Geletta: the ratio of the incidence of a dz in people who are exposed to a risk to the incidence in people without exposure to risk. - RCTs or cohort studies - RR = 1 implies that the event is equally probable in both groups
How to calculate OR and RR?
Adverse Event + Adverse Event - Totals Therapy + A B A+B Therapy – (control) C D C+D Totals A+C B+D N (A+B+C+D) - OR = (a/b)/(c/d) - CER = C/(C+D) - EER = A/(A+B) - RR = EER/CER
Which of these is a feature of the mnemonic PICO? Patient, information, immunization, origin and place?
- Patient. PICO = patient/population, intervention, comparison intervention, outcome
Which of these study designs would be considered lowest in the generally accepted hierarchy of evidence? 1. RCT 2. Cross-sectional survey 3. Prospective cohort study 4. Meta-analysis 5. Retrospective case-controlled study
- Cross-sectional survey
Which of these is the best design for a study of therapy? 1. Case-controlled study 2. Double-blind RCT 3. Single-blind RCT 4. Cohort study 5. Cross-over trial
- Double-blind RCT
Studies of prognosis seek information on which of the following? 1. The reason a particular condition has happened 2. The results of a particular intervention 3. The outcome of a test for a condition 4. The possible outcomes of a disease or condition 5. Which patients survive longest
- Answer = 4
35 yo female to clinic w/ fresh dog bite. It appears clean. The patient does not like to take abx and asks if they’re necessary as you write the rx. Which of these components will help write an answerable question in this situation? 1. Woman with dog bite 2. Oral, prophy abx 3. No antibiotics 4. Wound infection 5. All of these will be useful
- PICO: patient/problem/population, intervention, comparable intervention, outcome - 5. All of these will be useful
Which of these sources would be most likely to provide the best information? 1. Journal of ID 2. PubMed 3. Harrison’s Internal Medicine 4. Current NEJM 5. A current textbook of ID
- PubMed
Best source for systematic reviews
- The Cochrane collaboration
68 yo male to clinic c/o painful left shoulder for several weeks. In past, injection of corticosteroids into shoulders has worked well for your patients. However, you noticed a paper on injecting steroids for tennis elbow has shown short-term improvement but worse long-term outcomes than watchful waiting. You are uncertain how to proceed. 1. What is a useful, answerable question in this situation? a. Do corticosteroids improve shoulder pain? b. In a 68 yo male with shoulder pain, will injection of a corticosteroid into the shoulder result in improvement? c. What is the course of shoulder pain in an elderly man who receives corticosteroids? d. How can shoulder pain best be relieved? e. In patients over 60, does shoulder pain improve with or without corticosteroids?
- E
45 yo healthy male comes to clinic for a routine examination. He has noted no problems during the past year. A part of his PEX includes routine UA. The test shows microscopic quantities of blood. You are worried about the finding but uncertain whether or how to proceed. Give an example of an answerable questions.
- P: male patient with microscopic hematuria - I: renal ultrasound and cystoscopy - C: watchful expectancy - O: no urinary tract disease
Which of the questions can be successfully answered by implementing a statistical technique? A. What results if one repeatedly puts blue litmus paper into acid solutions? B. What results if one repeatedly flips a fair coin? C. What is the acceleration rate of an object that is at constant velocity? D. All of the above questions can be effectively answered using a statistical technique.
- B
Ordinal variables have values A. That differ in name only B. That differ in magnitude C. In which the intervals between values are equal in size D. Both B and C
- B
Which one of the following may be classed as an interval variable? A. IQ B. Height in centimeters C. Hair color D. Percentage correct
- A
Inferential statistical techniques are used to describe data that come from entire populations and so population notation is used. This statement is incorrect because: A. If we have data on the entire population then we do not need inferential statistics. B. When describing entire populations, sample notation should be used. C. None of the statements given here correctly describe why the statement is wrong. D. Entire populations are too large to describe. E. Actually, the statement is correct.
- A
A researcher discovers that age and income are related in a systematic way. To describe this relationship, we would probably use a: A. clinical technique B. predictive technique C. correlational technique D. collateral technique
- C
Suppose you collect data on your patients and want to find out where most of them live. You have a variable in your data that is named “Place of Residence” with the following values: 1=West DSM, 2=East DSM, 3=North DSM, 4=South DSM. Which of the following graphs should be used to give you the correct answer? A. histogram B. frequency polygon C. ogive D. bar graph
- D
Data: 2, 12, 12, 27, 27, 31. This distribution: A. is unimodal B. is bimodal C. has no mode D. has a mode of 19.5
- B
Which measure of central tendency could be determined by glancing at a bar graph? A. mean B. median C. mode D. all of the above
- C
For a given distribution of scores, you are told that Σ(X - μ) = 2. Which of the following is most likely true? a. a few scores are very far above the mean b. this must be a negatively skewed distribution c. this must be a positively skewed distribution d. an arithmetic error was made
- D
In which of the following would you NEVER calculate the median? a. positively skewed distribution b. a normal distribution c. a distribution with a few very extreme scores at the upper end d. an open-ended distribution e. a distribution of a nominal variable
- E
The mean on a test is 109 and s=0. This show that: a. the distribution is rectangular b. everyone got the same score c. the variable is discrete d. only one person took the test
- B
The standard normal curve a. has relative frequency along the ordinate – which add up to a unity b. can have a mean of 50 c. has a mean = median = mode d. both A and C e. all of the above are correct
- D