Critiquing Research Articles

Question 1

WHy do we critique literature?

Answer 1

To evaluate published PE research than conduct studies

Question 2

Appropriateness of study methods depends on…?

Answer 2

1. Specific question or objective
2. Study perspective
3. Time period necessary to determine outcomes of the alternatives
4. Resources available to researchers

Question 3

What are the questions for critiquing research articles?

Answer 3

1. Complete Title: Is the title of the research article appropriate?
2. Clear Objective: Is the objectives of the research article clearly stated
3. Appropriate Alternatives: Were appropriate alternatives/comparators considered in the study?
4. Alternatives Described: Was a complete description of the comparable alternatives provided?
5. Perspective Stated: Is the perspective of this study clearly provided?
6. Type of Study: Is the type of study clearly provided?
7. Relevant Costs: Were all relevant costs included in this study?
8. Relevant Outcomes: Were all relevant outcomes included in this study
9. Adjustments or Discounting: Were adjusting or discounting appropriate in this study?
10. Reasonable Assumptions: Were assumptions in this study clearly stated and reasonable?
11. Sensitivity Analyses: Were sensitivity analyses conducted for important estimates in this study
12. Limitations Addressed: Were limitations clearly addressed in this study
13. Appropriate generalizations: Were generalizations beyond the study population appropriate?
14. Unbiased conclusions: Did the authors present an unbiased summary of the study results?

Question 4

What is a complete title?

Answer 4

1. What type of study was performed
2. Title biased?
3. Can the evaluator determine what products/services are compared in the study?

Question 5

Why do we need clear objectives?

Answer 5

1. Did the researchers provide a well-defined question in an answerable form to its readers?
2. Ideal to clearly state the objective at the beginning of the journal article

Question 6

Why do we need appropriate alternatives?

Answer 6

Ideal for researchers to compare the most effective treatment alternatives available

Question 7

Why do we need alternatives described??

Answer 7

Could the research in this study be replicated by another group based on the info provided

Question 8

Why do we need perspective stated?

Answer 8

The study perspective will inform the reader about whose costs are measured in the study

Important for readers to determine the study perspective, as it helps to determine whose costs are being evaluated in the study

Question 9

Why is type of study important?

Answer 9

Clearly providing the type of study used by researchers will help their readers to more easily follow the methodology, results, and conclusions of the study

Question 10

Why is relevant cost important?

Answer 10

1. Were all appropriate types of costs evaluated?
2. Were costs collected for an appropriate time frame?
3. Did the article reference their sources, if the costs were estimated from outside materials?
4. Did the article justify why certain important costs were not included in the study?

Question 11

What are costs that are excluded?

Answer 11

1. Protocol driven costs
2. Identical costs for all alternatives in the study

Question 12

What are protocol drivent costs?

Answer 12

Costs that occur when following the research protocol of a RCT, but would not occur in “real-world” clinical practice

Question 13

What are questions for relevant outcomes?

Answer 13

1. Are the outcomes measured in the study considered relevant clinical outcomes that are important to healthcare providers?
2. Were outcomes measured for an appropriate time frame?

Question 14

What are questions for adjustments or discounting?

Answer 14

1. Was adjusting or discounting appropriate for this study?
2. If appropriate, were any adjustments or discounting performed in this study?

Question 15

What are reasonable assumptions?

Answer 15

Estimates of costs and outcomes are common
1. Possible that estimates are not precise
2. Possible that estimates used in research are not agreed upon its readers

Are the assumptions made in the study reasonable in the context of their clinic practice or their decision-making process?

Question 16

What is sensitivity analyses?

Answer 16

Determines how the results of an analysis would change if the assumptions are varied over a relevant range of values

Question 17

What is the purpose of sensitivity?

Answer 17

Determines if the results of the study analysis are robust:
1. If same option is chosen for the full range in sensitivity analysis, the analysis is insensitive or robust to this range of values, which adds confidence to the study results.
2. If the alternative choice changes based on range of values, the analysis is sensitive to this estimate, which should trigger caution to the reader when interpreting results.

Question 18

What are limitations addressed?

Answer 18

Important to address important limitations of the study

Question 19

What are ex of limitations addressed?

Answer 19

1. Using retrospective databases -> increased risk of selection bias
2. Using a specific population -> limits generalizability of study results
3. Using a small sample size -> limits statistical comparisons
4. No data available for certain costs or outcomes -> assign broad assumption for costs

Question 20

What is appropriate generalization?

Answer 20

If results are based on data from a specific patient population that is atypical to the general patient population, researchers should advise the readers to avoid extrapolating the data to general population.

Question 21

What is an unbiased conclusion?

Questions to ask?

Answer 21

1. Did the authors choose appropriate therapy alternatives for this study?
2. Did the authors use unbiased, reasonable estimates of costs & outcomes when determining study results?
3. As a future pharmacist & healthcare provider, do you support the findings of this study?
4. Does the results of this study make sense to you?