Research Evaluation Block 1 Flashcards
Define Prevalence
Number of occurrences at one particular time
Obtained from cross-sectional studies, no timeline, only snap shot in time
Define Incidence
Occurrence, rate, frequency of disease
Obtained from Cohort Studies and follows cohort through time
Define Correlation Coefficient
“r”
Absolute value of coefficient telling how strong relationship is between variables
-1 - 1, closer to them is stronger relationship
Closer to 0, weaker relationship
Define Relative Risk
Likelihood an event will occur w/in a population
Experimental Event Rate/Control Event Rate
Makes insignificant findings APPEAR significant
Define Sensitivity
People w/ disease compared to people with a positive test for the disease
(tests ability to identify correctly who have the test)
Rules out disease
Define Specificity
People without disease who have a negative test
(ability of test to correctly ID those who don’t have the disease)
Rules disease in
Define Two-Stage testing
Calculates net sensitivity and net specificity of using both tests in sequence
After completion, loss of net sensitivity and gain of specificity
Define Simultaneous Testing
PT is positive if tested pos on one or more tests
PT is negative if tested neg on ALL tests
Net gain in sensitivity and loss of specificity
Define External Validity
Ability to apply results obtained from studied population to broader populations
AKA generalization
Define Internal Validity
W/in confines of study, results appear to be accurate and investigators interpretation is supported
Define Confidence Interval
More important than p-value and a better determination of significance
95% CI= 95% certain that true value is within the CI range, narrower the better
Define Number Needed to Treat
of PTs needed to receive new intervention instead of standard alternative in order for ONE additional patient to benefit, used for prophylactic measures
Very effective treatments range: 2-4
10 or less for therapy
20 or less for prevention
Define Number Needed to Harm
Treatment is detrimental
Will have negative absolute risk reduction
Define P-Value
Probability level, chance of random error
Likelihood that difference observed between two interventions happened by chance
Define Efficacy Trial
Attempt to learn if drug, procedure or program works under ideal conditions
Define Type I Error
Reject the null when null was true
Finding an effect that isn’t real
Convicting innocent man to prison
Considered worse than Type II
Define Type II Error
Failure to reject null hypothesis
Missing an effect that does exist
Not convicting guilty man
Define Confounding
Variables that correlate in/directly with in/dependent variable
Define Case Report
Literature identifying single incident and discussing pertinent factors related to PT
Define Case Series
Study analyzes number of individual cases that share a commonality, usually w/ relatively low number of subjects
Define Case-Control
Studies w/ PTs already w/ condition/case compared to those who don’t have it (control)
Researcher looks back
Define Systematic Review
Thorough, comprehensive, explicit way of interpreting medical literature
Define Meta-analysis
Method for combining study data from several studies to justify quantitative summary to develop single conclusion with greater statistical power
Define Prospective Cohort Study
IDs group of PTs already taking treatment/have exposure and follows them forward over period of time
Define Retrospective Cohort Study
Starts w/ cohort and goes back in time to evaluate past exposures to risk factors
Define Cross-sectional Study
Examines relationship between outcome and other variables of interest at one particular time
Determines prevelance
Cannot show casuality
Does not establish temporal relationship
Define Quasi-experimental study
Non-randomized control study
Control group predetermined w/out random assignment and compared to control group
Define Case-Crossover study
Type of Case-Control study
Each case becomes own individual control
Used for transient exposures
Define Measures of Central Tendency
Mean- average, sum divided by number in set
Median- middle point
Mode- score occurring most frequently; least precise measure; bimodal distribution- 2 most common values
Define Positive Predictive Value
Define Negative Predictive Value
Pos- proportion of PTs who have disease and a pos test
Neg- PTs who do not have disease and a negative test
Define Likelihood Ratio
Likelihood of someone with disease/likelihood someone without disease
W/WO
Define Odds Ratio
Measures strength of association between exposure and disease
Define Epidemiology
Study of distribution and determinants of health-related events in populations and application of the study to control health problems
Define Biostatistics
Branch of stats dealing with data relating to living organisms
Difference between Descriptive and Analytical Research?
Descriptive:
observational
patterns of disease occurrence
generates hypothesis
Analytical:
analyzes
investigates relationships
tests hypothesis
Examples of descriptive studies?
Individual:
Case reports
Clinical series
Population:
Ecologic studies
Define Explanatory Studies
Attempt to provide insight into etiology/determine better PT outcome
What are the two types of Explanatory studies?
Experimental
Observational
Define an Experimental Explanatory Study?
Give examples
Active intervention from investigator Controlled trial Clinical trial Educational intervention Healthcare trial Intervention trial
Define an Observational Explanatory study?
Give examples
Investigator observes nature Case control Follow up Cross sectional Cohort
What are the layers of Quality of Evidence Pyramid?
Meta Analysis Systematic Review Critically Appraised literature Evidence Based Practice Randomized Control Trials Non-Random Control Trials Cohort Studies Case series/studies Individual case reports Background info/expert opinion/non-EBM guidelines
What are the two types of Analytic Descriptive studies?
Experimental:
Clinical trial
Community trial
Education intervention
Observational:
Case control
Cohort f/u
Cross sectional
What are Case Series used for?
Examine adverse events/effects
Catalog new diseases/outbreaks
Determine feasibility/safeguards of new treatment/intervention
Potential efficacy of new treatment
What do both Case Reports and Case Series lack?
Sufficient methodological rigor
Evidence not applicable to large populations, may be circumstantial or confounding factors were present
Purpose of Descriptive Study Ecologic Study
Determine relationship between exposure and disease among a population, not just individuals
What are the strengths of Cross Sectional Studies?
Assesses multiple outcomes/exposures simultaneously
Completed quickly
Data gathered leads to further studies
Generates Prevalence
What are the limitations of Cross Sectional Studies
No time reference- "snapshot" Common conditions only Doesn't calculate incidence, is a prevalence study Results dependent on study population Only useful for common conditions
What type of study may follow a case series study?
Case-control study, retrospective look at causes
What are the strengths of Case Control studies?
Good for Rare Outcomes Evaluates many exposures Ideal for ideal/explanatory ides Simple/fast/efficient Inexpensive
What are the limitations of Case-control studies?
Single outcome
High risk of bias/confounding variables
No prevalence/incidence/relative risk
Define Nested Case-Control study?
A case control study
Large enrollment study within large cohorts for people at risk of disease/outcome
Define Case-Cohort
Same as nested case-control but with controls randomly chosen at beginning of study
Define Cohort
Groups of people who share common characteristic/experience and remain in a group for period of time
What is the strongest observational study?
Cohort Study
Potential biases in Case Control Studies
Selection bias Information bias Researcher/observer bias Voluntary response bias Matching Multiple controls
Potential biases in Cohort studies?
Selection bias- lost to f/u
Information bias- observer bias
What are the strengths of cohort studies?
Studies multiple effects of single exposure
IDs temporal relationship between exposure and outcome
Confirms cause/effect and the magnitude
Measures incidence
Calculates relative risk
What study has the highest validity of observational study design?
Cohort study
What are the limitations of Cohort Studies?
Expensive/time consuming Ineffective for rare diseases Lose participant f/u Risk of confounding variables Presence of records/recall
What is the purpose of randomization?
Prevent potential bias by investigator, strives for comparability
The term “controlled” in randomized control studies implies predefined 6 things
Hypothesis Endpoint to address hypothesis Enrollment/ f/u methods Eligibility/exclusion criteria Rigorous monitoring Analysis plan/stop rules
Why are randomized control studies controlled?
Eliminate confounding variables
Minimize bias
Define single, double and triple blind
Single- concealed from one group
Double- concealed from both groups
Triple- concealed from subjects, administers and interpreters
What is the Gold Standard?
Double blinded Randomized Control Trial, minimizes chance of bias
Huge strength of randomized control studies
What are the limits of Randomized Control Studies?
Large trials Long term f/u Compliance Expensive Ethics
What does “Primum Non Nocere” mean?
First do no harm
What are the limitations of Non-randomized control studies?
Study group characteristics not balance w/ baseline confounding study’s results
What are the 4 major areas of methodological concern in Non/randomized control studies?
Enrollment
Allocation
F/u
Analysis
What parts of the EBM pyramid represent the strongest evidence?
Top 2
Systematic reviews
Meta-analyses
What are the strengths of systematic reviews?
Review of current literature Less costly Less time Generalized results Reliable/accurate than individual studies Evidence-based resource
What are the limitations of Systematic reviews?
Time/labor consuming
Not easy to combine studies
What is the meta-analysis conclusion statistically stronger than any single study?
Increased subject numbers
Diversity
Accumulated effects/results
What are the limitations of meta-analysis?
Difficult/time consuming Lack of adequate data Advanced statistic techniques Hetergeneity Non-reproducible results Publication/Selection/Misclassification bias
Define Variance
Quantifies amount of variability around mean
Define Standard Deviation
Measure of variation of scores about the the mean
What is the empirical rule for data when discussing standard deviation?
68-95-99
Only applicable to bell curves
68% fall within 1 standard deviation of mean
95% fall within 2 standard deviations of mean
99.7% fall within 3 standard deviations of mean
Scatterplots are useful summary of sets of ______ data
Bivariate- two continuous variables
Visual relationship between two variables and helps interpret correlation coefficient or regression model
What type of graph/visual grouping aids w/ interpreting correlation coefficient/regression model?
Scatter plots
Define Pearson Correlation and what it’s associated with?
Correlation coefficient
Most common measure of association
Misleading if relationship is non-linear
Define Alternative Hypothesis
AKA Research Hypothesis
Establishes expected relationship between variables being tested
Define Null Hypothesis
Statement of no difference/relationship between variables
When is the Null Hypothesis accepted/rejected?
Statistical difference is present, alternative is accepted and reject null hypothesis
If no statistical difference is present, researchers retain/fail to reject null hypothesis
Define Significance
Probability of rejecting a true Null Hypothesis
.05= 5:100 times, Null wold be falsely rejected; 5% chance results happened by chance
What 3 criteria must exist to conduct a screening test?
Target disease is important cause of mortality/morbidity
Proven/acceptable tests exist to detect early stage of disease
Treatment is available to prevent mortality/morbidity once results are known
Pros/cons of Sensitivity Testing
Screening test ability to ID disease presence
High sensitivity= few PTs w/ disease will be missed
Rules out disease, low probability of false negative
Pros/Cons of Specificity
Tests ability to truly ID absence of disease
Rules disease IN
High specificity, low false positive
When are high sensitivity and high specificity tests useful to clinicians?
Sensitivity- when test is neg
Specificity- when test is pos
What is the use/benefit of regression analysis?
Method of predicting change in dependent variable by changing one or more independent variables
What are the 4 types of data?
Categorical: nominal/ordinal
Continuous: interval/ratio
Define Nominal and Ordinal Data
Categorical data
Nominal: categories w/ no order (gender, race, blood type)
Ordinal: sequence/ranked (small to large, light to heavy, easy to difficult)
Define Interval and Ratio
Continuous data
Interval: interval along scale (temperature)
Ratio: presence of an absolute zero on scale is present (weight)
What is the benefit of using a Chi-Square test?
Most common ways to examine relationships between two or more categorical variables
Does not give info on strength of relationship
Define Control Event Rate
How often particular event occurs w/in scientific control group
Define Experimental Event Rate
How often particular event occurs w/in experimental group
What type of cohort is cumulative incidence used in?
Fixed
What type of cohort is incidence rate used in?
Open
Define Ecological Fallacy
Bias specific to ecological studies
Occurs when relationships for groups are assumed to be true for individuals
Case-control studies may follow what type of study?
Case-series as retrospective look at cause
What study attempts to capture cause and effect relationship by comparing frequency of risk factor among the exposed/not exposed
Case control study
Define selection bias
inappropriate selection for case studies/controls
Cases: selected from hospital/clinic/registry, single source selection may not be generalized to all PTs
Control: controls from same reference population as cases
Define Information Bias
Recall/subject bias occuring when there is difference in recalling info of exposure
Define Researcher/Observer Bias
When researcher/observer evaluates cases vs controls differentially
Define Voluntary Response Bias
Case subject think they’ve been exposed respond at higher rate to control
Define matching
Process of selecting controls so they are similar to cases in characteristics (age, race, sex, occupation)
What type of investigation is a cohort study?
Epidemiologic
When must the criteria for determining selection be specified?
Before study has begun
What is the best approach in the design of a trial?
Randomization
What is the critical element of randomization?
Unpredictability of the next assignment
How/what are methods of randomization accomplished?
Computer programs
Envelope system- only opened after consent is given and subject meets eligibility criteria
Define stratified randomization
Used when concern about comparability of groups and one/few important characteristics
Stratifies study population by each variable considered important then randomized into treatment groups
Define blinding
Why is it used?
Concealment of group allocation from one/more individuals involved in clinical research study
Used to make sure knowing type of treatment doesn’t effect participants response to treatment, healthcare providers behavior, or assessment of treatment effects
When is blinding usually used?
When comparing two or more types of interventions
Define Planned Crossover
After time on certain therapy, PT is switched to alternate therapy
PT serves as own control, but must have washout period
Define Unplanned Crossover
Subjects are randomized cross-over to other group, randomization is broken/lost
What is the effect of non-compliance?
Reduce any observed differences since treatment group will include some who didn’t receive therapy
Systematic reviews and meta-analysis are both subject to bias based on what?
Inclusion/exclusion criteria
What is the difference between systematic review and meta-analysis?
Review- thorough comprehensive way of interpreting medical literature
Analysis- statistical approach to combine data from selected studies
Systematic reviews _____ and ____ all empirical evidence to answer a research question
Appraise
Synthesize
What are the strengths of meta-analyses?
Greater statistical power
Confirmatory data analysis
Greater ability to extrapolate to general population
Define Positive correlation on scatter plot
Define negative correlation on scatter plot
Pos- X inc, Y inc
Neg- X inc, Y dec
Pearson’s correlation is very sensetive to what?
Outlying values
Define Spearman Correlation
Non-parametric version of pearson’s correlation
Calculation based on ranks of data points of x/y values
When is the alternative hypothesis accepted?
Statistically significant difference
Accept alternative, reject null
Define power analysis
Calculates number of participants a study must have to draw accurate calculations
What happens when p-value is less than/equal to .05?
What happens when p-value is greater than .05?
Reject Ho, results are statistically significant
Fail to reject Ho, results are not significant
What does it mean if a CI includes 1 (null value)?
Results are clinically insignificant
What does high sensitivity mean?
What does high specificity mean?
Low probability of false negative
Low probability of false positive
What happens to the Positive Predictive Value with low prevalence?
Lower PPV
False positives increase
Less reliable pos test results
What happens to Negative Predictive Value with low prevalence?
Higher NPV
False neg test decreased
Neg test is more reliable
What calculation can be done to determine the probability of a disease in a low prevalence setting?
Likelihood ratio
Define positive and negative likelihood ratio
Pos- Pos test (sensitivity) / non-diseased (specificity)
Rules disease in
1-infinity, desirable is 5 or more
Neg- diseased people with neg test (sensitivity)/non-diseased people with neg test (specificity)
Ruling out disease
0.0-1.0
Smaller the better, desirable is 0.2 or less
How is relative risk calculated?
Cumulative incidence data to measure probability of developing disease
Experimental Event Rate / Control Event Rate
EER/CER
Define T-test
AKA student’s t-test
Used to analyze continuous data by comparing means and standard deviations of two populations, computes a p-value to test null-hypothesis
What are examples of T-tests?
Histogram
Statistical tests
Define reliability
Degree to which measurements are reproducible
Define Independent Variable
Condition/intervention that predicts an outcome
Age/gender/marital status
Experimental treatment doesn’t change the value
Define Dependent Variable
AKA Outcome variable
Result that is presumed depending on independent variable
Definition of Evidence Based Medicine
Systematic approach to clinical problem solving allowing integration of best variable research evidence with clinical expertise and patient values
What are the 3 fundamental principles of EBM
Optimal clinical decision making requirements
Deciding if evidence is more/less trustworthy
Evidence alone is never sufficient enough to make decisions
What is the simplified process of EBM?
Assess Ask Acquire Appraise Apply
What are the 5 types of clinical questions?
Therapy Harm Differential Diagnosis Prognosis
Define Quantitative Research
Research w/ formal, objective info about the world with math quantification
Used to describe test relationships and examine cause/effect relationships
Define Qualitative Research
Research with phenomena that are difficult/impossible to mathematically quantify
What study design does not have a comparison group?
Case Series Study
What analytical design is simultaneous evaluation of exposure and outcome?
Cross Sectional Study
What type of study design can explore potential relationships between exposure and disease between two communities?
Ecological
What observational study design provides best casual estimate of relationship between exposure and outcome?
Prospective Cohort Study
What observational study allows designer to explore prevalence in a population?
Cross Sectional
What study design are control and cases selected/identified within framework of a cohort study?
Nested Case-Control
Which study design can calculate relative-risk?
Prospective cohort
Which study design can calculate incidence?
Prospective cohort
Which study design can not calculate odds ratio?
Case Report
Which study design is the participant’s disease/outcome free at the beginning of the study period?
Prospective Cohort
In general a p
value > 0.05 represents which of the following?
A statistically insignificant result
Which of the following p
values rejects the null hypothesis if the α value is set to 0.05?
P = 0.01
Which of the follow statement best defines a type II error?
A type II error occurs if we fail to reject the null hypothesis when the null hypothesis is false
Which of the following terms describes the probability of correctly rejecting the null hypothesis?
Power
Which of the following correlation coefficients represents the strongest positive correlation between two variables?
0.99
Which of the following best describes a p value ≤0.05?
Means there is a 5% chance that the results occurred by chance alone
Which of the following statements best describes an alternative hypothesis?
The alternative hypothesis represents the expected relationship between the independent and dependent variable
Which of the following represents the most common measure of variability in research?
Standard Deviation
Which of the following is a measure of central tendency?
mean
Which of the following is the least precise measure of central tendency?
Mode
What type of error is made when we reject the null hypothesis when the null hypothesis is true?
Type I error (α error)
Which characteristic of a diagnostic test is most helpful in ruling-in a disease if the result of the test is positive?
Specificity
Which of the following test characteristics defines a tests ability to correctly identify the presence of disease?
Sensitivity
Which of the following is the most valuable in determining the statistical significance of an effect estimate?
Confidence interval
In which of the following study designs can incidence be calculated?
Cohort study
Which of the following is an example of a categorical variable?
Gender
Which of the following is an example of a continuous variable?
BP
The term that describes the probability of patients who test positive and truly have the disease is:
Positive Predictive Value
Sequential (two-stage) testing results in which of the following?
Loss in net sensitivity and net gain in specificity
Simultaneous testing results in which of the following?
Net gain in sensitivity and net loss in specificity
Which of the following terms best describes the current disease burden in a community and population?
Prevalence
Assuming a disease has no cure and low risk of subsequent death. What is the likely effect on prevalence if an aggressive screening program is initiated?
Prevalence is likely to increase
A prospective cohort study exploring the relationship between coffee consumption and Gastric Cancer in a large urban city found that the risk of gastric cancer was higher among coffee drinkers; however, the researchers did not adjust for potentially confounding variables. Is the inference that coffee drinkers are at greater risk of developing Gastric Cancer likely to be correct or incorrect?
Incorrect as there may be other relevant factors that may explain this relationship
Which of the following represents the highest level of evidence based research?
Meta-analysis of Randomized Controlled Trials
Which observational study design provides highest level of validity?
Prospective Cohort Study
Which of following is the best method of determining if randomization was successful when conducting a randomized controlled trial?
Confirm that researchers did not alter the allocation participants into treatment or control groups
You work in a family practice clinic and order a rapid strep test for a patient with a sore throat. The rapid strep test has a sensitivity of 90% and specificity of 98%. What does a positive test likely represent?
The patient likely has an infection
The current rapid strep test has a sensitivity of 90% and specificity of 98%; however, a new generation test has a sensitivity of 99% and a specificity of 98%. What is the likely result utilizing the new test?
The new test will identify more people with an infection
The results of study reports a relative risk (RR) of 2.50 (95% CI: 1.50-3.50). Which of the following is best interpretation of the RR and 95% CI?
The reported RR is statistically significant
Which of the following is most appropriate definition of a confounder?
A variable related the factor and outcome, but not in the causal pathway
What is the relation between a Positive Predictive Value and prevalence?
What is the relation between a Negative Predictive Value and prevalence?
PPV- directly related to prevalence
NPV- Inversely related to prevalence
