WEEK 2 - CRITIQUING LITERATURE Flashcards
3 things to consider when assessing the quality of the study and how useful it is
- Results (what did the study find?)
- Validity (do the results match the conclusions?)
- Applicability (will they help you with your own clients or patients?)
What is the IMRAD forma t
- Introduction (why the authors decided to do this particular piece of research)
- Methods (how the did it, and how they chose to analyse their results)
- Results (what they found)
- Discussion (what they think the results mean)
Major aspects you need to consider when reading an original (primary) research paper are:
- the sample/participants
- the setting
- how data is collected
- how it is analysed
Reviewing the methods of primary research papers
- the sample/participants
- the setting
- how data is collected
- how it is analysed
Sample and setting
- A strong sample is vital for a successful research project.
- Participants in a study may differ from real-life patients (age, gender, co-morbidities etc.) so it is important the sample is close enough to your own patients or clients to make the results applicable.
Questions you could ask include
- Who was included in and excluded from the study?
Sample and setting Questions you could ask include
- How were the participants recruited?
- Did the participants receive any special care apart from any intervention that is the focus of the study?
- Where did the study take place?
- Did the study have ethical clearance?
What data collection methods wereused?
- Research design will dictate the way data should be collected (e.g. specific tools or instruments, or focus groups)
- General questions to ask include:
1. What data was collected?
2. Would this be sufficient to answer the question?
What data collection methods were used? Questions you could ask include
- Who collected it and how? If a tool was used, was it validated and appropriate to the design?
- What outcome was measured or explored?
- How was bias avoided or minimised? How did the researchers strengthen the validity/reliability or credibility/dependability of their study?
Analysing quantitative data
- Quantitative results are given in numbers and produced through statistical analysis, whether descriptive or inferential
The main questions for you to ask are:
- Was the study large enough, and continued for long enough, and was follow‐up complete enough, to make the results credible?
- Was assessment ‘blind’ or ‘masked’ to avoid performance bias?
- Are the results statistically or clinically significant?
Analysing qualitative data Questions you could ask include
- Analysing qualitative data is more subjective than analysing quantitative studies. You are looking for enough description of the analysis process to convince you the research is credible.
- Has the phenomenon been described fully?
- Has the researcher examined their own assumptions/role in the research?
- Have the themes, structures or processes been explained, and are they clear and convincing? Has more than one person been involved in the coding process, to strengthen the analysis?
- Have validation techniques been used, such as triangulation, member checking, etc.?
- Are they supported by evidence in the form of participant quotations?
Reviewing the methods of secondary (review) papers
- A systematic review is an overview of primary studies
- For EBP, the most important and useful review is the systematic review, with or without meta_analysis
- Meta-analysis = overview of quantitative studies
- Meta-synthesis = overview of qualitative studies
Why is a systematic review so useful in EBP?
- overview of primary studies
- asked a focused question on a specific clinical problem
- uses prescribed standard (PRISMA)
- clear (objectives, sources and methods) and rigorously conducted (explicitly, transparent, reproducible)
Level of evidence, highest to lowest
- systematic review
- rct
- cohort studies
- -case control studies
- case series
- case reports
- ideas, editorials, opinion
3.3 Reviewing the methods of secondary (review) papers
- Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) statement: the agreement on a standard, structured format for writing up and presenting systematic reviews.
- Following these structured checklists makes systematic reviews and meta‐analyses a whole lot easier to find your way around.
PRISMA checklist
Questions based on the PRISMA checklist (but greatly shortened and simplified) to ask about any systematic review of quantitative evidence include:
- What is the focused clinical question that the review addressed?
- Was a thorough search carried out of the appropriate database(s), and were other potentially important sources explored?
- Who evaluated the studies, and how?
- How sensitive are the results to the way the review has been performed?
- Have the numerical results been interpreted sensibly, and are they relevant to the broader aspects of the problem?
Meta-analysis continued
- The reader can compare both the methods and the results of several trials to arrive at a synthesised result.
- Similar syntheses of qualitative studies, known as meta_syntheses, have begun to be produced.
- More difficult to carry out due to more varied methods and theoretical approaches but can provide useful insights that help to overcome the problem of small samples and limited transferability.
Meta‐analysis for the non ‐statistician
- Meta‐analysis: merges the results from a number of single, independent quantitative studies, to produce an overall or ‘absolute’ result that provides stronger evidence than the original small studies could do.
The meta‐analyst is looking for homogeneity.
They decide which of the outcome measures chosen by the authors of the primary studies is the best one/s to use in the overall synthesis, and then extract these from all the papers.
Tabulate other information (e.g. inclusion and exclusion criteria, sample size, withdrawal (‘drop‐out’) rate), and results of primary and secondary end‐points of all the studies included.
