Critiquing Research Articles Flashcards
WHy do we critique literature?
To evaluate published PE research than conduct studies
Appropriateness of study methods depends on…?
- Specific question or objective
- Study perspective
- Time period necessary to determine outcomes of the alternatives
- Resources available to researchers
What are the questions for critiquing research articles?
- Complete Title: Is the title of the research article appropriate?
- Clear Objective: Is the objectives of the research article clearly stated
- Appropriate Alternatives: Were appropriate alternatives/comparators considered in the study?
- Alternatives Described: Was a complete description of the comparable alternatives provided?
- Perspective Stated: Is the perspective of this study clearly provided?
- Type of Study: Is the type of study clearly provided?
- Relevant Costs: Were all relevant costs included in this study?
- Relevant Outcomes: Were all relevant outcomes included in this study
- Adjustments or Discounting: Were adjusting or discounting appropriate in this study?
- Reasonable Assumptions: Were assumptions in this study clearly stated and reasonable?
- Sensitivity Analyses: Were sensitivity analyses conducted for important estimates in this study
- Limitations Addressed: Were limitations clearly addressed in this study
- Appropriate generalizations: Were generalizations beyond the study population appropriate?
- Unbiased conclusions: Did the authors present an unbiased summary of the study results?
What is a complete title?
- What type of study was performed
- Title biased?
- Can the evaluator determine what products/services are compared in the study?
Why do we need clear objectives?
- Did the researchers provide a well-defined question in an answerable form to its readers?
- Ideal to clearly state the objective at the beginning of the journal article
Why do we need appropriate alternatives?
Ideal for researchers to compare the most effective treatment alternatives available
Why do we need alternatives described??
Could the research in this study be replicated by another group based on the info provided
Why do we need perspective stated?
The study perspective will inform the reader about whose costs are measured in the study
Important for readers to determine the study perspective, as it helps to determine whose costs are being evaluated in the study
Why is type of study important?
Clearly providing the type of study used by researchers will help their readers to more easily follow the methodology, results, and conclusions of the study
Why is relevant cost important?
- Were all appropriate types of costs evaluated?
- Were costs collected for an appropriate time frame?
- Did the article reference their sources, if the costs were estimated from outside materials?
- Did the article justify why certain important costs were not included in the study?
What are costs that are excluded?
- Protocol driven costs
- Identical costs for all alternatives in the study
What are protocol drivent costs?
Costs that occur when following the research protocol of a RCT, but would not occur in “real-world” clinical practice
What are questions for relevant outcomes?
- Are the outcomes measured in the study considered relevant clinical outcomes that are important to healthcare providers?
- Were outcomes measured for an appropriate time frame?
What are questions for adjustments or discounting?
- Was adjusting or discounting appropriate for this study?
- If appropriate, were any adjustments or discounting performed in this study?
What are reasonable assumptions?
Estimates of costs and outcomes are common
1. Possible that estimates are not precise
2. Possible that estimates used in research are not agreed upon its readers
Are the assumptions made in the study reasonable in the context of their clinic practice or their decision-making process?
What is sensitivity analyses?
Determines how the results of an analysis would change if the assumptions are varied over a relevant range of values
What is the purpose of sensitivity?
Determines if the results of the study analysis are robust:
1. If same option is chosen for the full range in sensitivity analysis, the analysis is insensitive or robust to this range of values, which adds confidence to the study results.
2. If the alternative choice changes based on range of values, the analysis is sensitive to this estimate, which should trigger caution to the reader when interpreting results.
What are limitations addressed?
Important to address important limitations of the study
What are ex of limitations addressed?
- Using retrospective databases -> increased risk of selection bias
- Using a specific population -> limits generalizability of study results
- Using a small sample size -> limits statistical comparisons
- No data available for certain costs or outcomes -> assign broad assumption for costs
What is appropriate generalization?
If results are based on data from a specific patient population that is atypical to the general patient population, researchers should advise the readers to avoid extrapolating the data to general population.
What is an unbiased conclusion?
Questions to ask?
- Did the authors choose appropriate therapy alternatives for this study?
- Did the authors use unbiased, reasonable estimates of costs & outcomes when determining study results?
- As a future pharmacist & healthcare provider, do you support the findings of this study?
- Does the results of this study make sense to you?
