The Hierarchy of Evidence Flashcards
1
Q
Hierarchy of Evidence
A
2
Q
Hierarchy of Evidence
A
3
Q
Assessing risk of bias
A
4
Q
Assessing risk of bias
A
5
Q
Assessing risk of bias
- Quality appraisal tools are used
- DURING …-… process – by authors to ensure high quality …
- AFTER …-… process - by readers to … risk of bias within studies
- Quality appraisal tools take the form of a … with a list of items that, for higher quality, should be included
- Within the study itself
- Within the report
- Some tools give rise to a … score or … rating
- Risk of bias can be assessed both
- Within individual studies
- Collectively acrossstudies
A
- Quality appraisal tools are used
- DURING write-up process – by authors to ensure high quality reporting
- AFTER write-up process - by readers to review risk of bias within studies
- Quality appraisal tools take the form of a checklist with a list of items that, for higher quality, should be included
- Within the study itself
- Within the report
- Some tools give rise to a numerical score or categorical rating
- Risk of bias can be assessed both
- Within individual studies
- Collectively acrossstudies
6
Q
Assessing risk of bias
- When are quality appraisal tools used?
A
- Quality appraisal tools are used
- DURING write-up process – by authors to ensure high quality reporting
- AFTER write-up process - by readers to review risk of bias within studies
7
Q
Assessing risk of bias
- Risk of bias can be assessed both…
A
- Within individual studies
- Collectively acrossstudies
8
Q
Assessing risk of bias
- Quality appraisal tools take the form of a checklist with a list of items that, for higher quality, should be included
- Within the study itself
- Within the report
A
- Within the study itself
- Within the report
9
Q
Case reports - Hierarchy of Evidence
- Disadvantages:
- Only …
- Lack of g…
- Uncontrolled
- Uncertain … o../s
- Changing features to preserve … could impact on usefulness
- Advantages:
- Can include …-… analyses/ experiential inquiries of person/ group in …-… setting
- Utility in emerging epidemics and previously unrecognised syndromes
- Fast and …
- Good … tool
A
- Disadvantages:
- Only descriptive
- Lack of generalisability
- Uncontrolled
- Uncertain future outcome/s
- Changing features to preserve anonymity could impact on usefulness
- Advantages:
- Can include in-depth analyses/ experiential inquiries of person/ group in real-world setting
- Utility in emerging epidemics and previously unrecognised syndromes
- Fast and cheap
- Good learning tool
10
Q
Case reports - Hierarchy of Evidence
- Disadvantages:
- Only descriptive
- … of generalisability
- …
- … future outcome/s
- Changing features to preserve anonymity could impact on …
- Advantages:
- Can include in-depth analyses/ experiential inquiries of person/ group in real-world setting
- Utility in emerging … and previously … syndromes
- … and cheap
- Good learning tool
A
- Disadvantages:
- Only descriptive
- Lack of generalisability
- Uncontrolled
- Uncertain future outcome/s
- Changing features to preserve anonymity could impact on usefulness
- Advantages:
- Can include in-depth analyses/ experiential inquiries of person/ group in real-world setting
- Utility in emerging epidemics and previously unrecognised syndromes
- Fast and cheap
- Good learning tool
11
Q
Animal trials - Hierarchy of Evidence
- Disadvantages:
- Not able to … all … features and …
- Introduction e.g. of allergens, pathogens not like …-…
- … between a given animal species and humans
- Animal … and …
- Advantages:
- Possibilities considered unethical in human trials
- Ability to control potential …
- Vast commonalities in the biology of most mammals
- Human diseases often affect other animal species
- Mechanisms and treatment also commonly similar in major infectious and non-infectious diseases
A
- Disadvantages:
- Not able to mimic all human features and phenotypes
- Introduction e.g. of allergens, pathogens not like real-life
- Differences between a given animal species and humans
- Animal protection and welfare
- Advantages:
- Possibilities considered unethical in human trials
- Ability to control potential confounds
- Vast commonalities in the biology of most mammals
- Human diseases often affect other animal species
- Mechanisms and treatment also commonly similar in major infectious and non-infectious diseases
12
Q
Animal trials - Hierarchy of Evidence
- Disadvantages:
- Not able to mimic all human features and phenotypes
- Introduction e.g. of allergens, pathogens not like real-life
- … between a given animal species and humans
- Animal … and …
- Advantages:
- Possibilities considered … in human trials
- Ability to control potential confounds
- Vast … in the biology of most mammals
- Human … often affect other animal species
- Mechanisms and treatment also commonly similar in … infectious and …-infectious diseases
A
- Disadvantages:
- Not able to mimic all human features and phenotypes
- Introduction e.g. of allergens, pathogens not like real-life
- Differences between a given animal species and humans
- Animal protection and welfare
- Advantages:
- Possibilities considered unethical in human trials
- Ability to control potential confounds
- Vast commonalities in the biology of most mammals
- Human diseases often affect other animal species
- Mechanisms and treatment also commonly similar in major infectious and non-infectious diseases
13
Q
Meta-analysis/Systemic reviews - Hierarchy of Evidence
- Disadvantages:
- Limited by … and amount of available …
- Meta-analysis of different study type occupies same top pole
- Limited inclusion of/ focus on … research
- …-defined question
- Advantages:
- Summary of overall evidence area
- … policy and clinical decision-making
- Identify and test sources of variability
A
- Disadvantages:
- Limited by quality and amount of available evidence
- Meta-analysis of different study type occupies same top pole
- Limited inclusion of/ focus on qualitative research
- Narrowly-defined question
- Advantages:
- Summary of overall evidence area
- Guide policy and clinical decision-making
- Identify and test sources of variability
14
Q
Meta-analysis/Systemic reviews - Hierarchy of Evidence
- Disadvantages:
- Limited by quality and amount of available evidence
- Meta-analysis of different study type occupies same top pole
- … inclusion of/ focus on qualitative research
- …-defined question
- Advantages:
- Summary of overall evidence area
- Guide policy and … decision-making
- Identify and test sources of …
A
- Disadvantages:
- Limited by quality and amount of available evidence
- Meta-analysis of different study type occupies same top pole
- Limited inclusion of/ focus on qualitative research
- Narrowly-defined question
- Advantages:
- Summary of overall evidence area
- Guide policy and clinical decision-making
- Identify and test sources of variability
15
Q
Evidence-based medicine (EBM)
- “Evidence based medicine is the c…, e…, and judicious use of current best evidence in making decisions about the care of … patients” (David Sackett)
- “(1) that our clinical and other health care decisions should be based on the best patient- and population-based as well as ..-based evidence;
- (2) that the … determines the nature and source of evidence to be sought, rather than our habits, protocols or traditions;
- (3) that identifying the best evidence calls for the integration of e… and b… ways of thinking with those derived from pathophysiology and our personal experience;
- (4) that the conclusions of this search and critical … of evidence are worthwhile only if they are translated into actions that affect our patients;
- (5) that we should continuously evaluate our … in applying these ideas.” (Sackett & Rosenberg)
A
- “Evidence based medicine is the conscientious , explicit, and judicious use of current best evidence in making decisions about the care of individual patients” (David Sackett)
- “(1) that our clinical and other health care decisions should be based on the best patient- and population-based as well as laboratory-based evidence;
- (2) that the problem determines the nature and source of evidence to be sought, rather than our habits, protocols or traditions;
- (3) that identifying the best evidence calls for the integration of epidemiological and biostatistical ways of thinking with those derived from pathophysiology and our personal experience;
- (4) that the conclusions of this search and critical appraisal of evidence are worthwhile only if they are translated into actions that affect our patients;
- (5) that we should continuously evaluate our performance in applying these ideas.” (Sackett & Rosenberg)
16
Q
EBM and hierarchies of evidence
- EBM in practice:
- Setting the …
- Finding the …
- … the Evidence
- … the Evidence
- … Performance
- A. Individual clinicians performing … appraisal of evidence as part of routine practice, using a … of evidence
- B. Individual clinicians being guided by guidelines/evidence synthesis formulated through … appraisal and review
A
- EBM in practice:
- Setting the Question
- Finding the Evidence
- Appraising the Evidence
- Applying the Evidence
- Evaluating Performance
- A. Individual clinicians performing critical appraisal of evidence as part of routine practice, using a hierarchy of evidence
- B. Individual clinicians being guided by guidelines/evidence synthesis formulated through hierarchical appraisal and review
17
Q
EBM and hierarchies of evidence
- EBM in practice:
- Setting the Question
- Finding the Evidence
- Appraising the Evidence
- Applying the Evidence
- Evaluating Performance
- A. Individual clinicians performing critical appraisal of evidence as part of routine practice, using a hierarchy of evidence
- B. Individual clinicians being guided by guidelines/evidence synthesis formulated through hierarchical appraisal and review
A
- EBM in practice:
- Setting the Question
- Finding the Evidence
- Appraising the Evidence
- Applying the Evidence
- Evaluating Performance
- A. Individual clinicians performing critical appraisal of evidence as part of routine practice, using a hierarchy of evidence
- B. Individual clinicians being guided by guidelines/evidence synthesis formulated through hierarchical appraisal and review
18
Q
EBM and evidence appraisal
- Evidence … is the process of deciding whether, and to what degree, evidence (either from a … study or an … base of many studies) supports a ….
A
- Evidence appraisal is the process of deciding whether, and to what degree, evidence (either from a single study or an evidence base of many studies) supports a claim.
19
Q
Hierarchy of evidence and health policy: Evidence-Based Policy (EBP)
- Health policy is developed based on
- Using the best … healthcare evidence overall
- To ensure … and equitable access to most effective and …-… healthcare
- Most research in medicine is produced by pharmaceutical and medical device companies – who want to later sell the product they make
- … is not-for-profit international organisation, which only collaborates with non-commercial organisations (80% of NICE and WHO guidelines use … Reviews)
- NICE guideline creation
- Independence from … and …
- …-… teams, stakeholder engagement and participatory process involving patients and carers
- Evidence … and transparent processes
A
- Health policy is developed based on
- Using the best quality healthcare evidence overall
- To ensure fair and equitable access to most effective and cost-effective healthcare
- Most research in medicine is produced by pharmaceutical and medical device companies – who want to later sell the product they make
- Cochrane is not-for-profit international organisation, which only collaborates with non-commercial organisations (80% of NICE and WHO guidelines use Cochrane Reviews)
- NICE guideline creation
- Independence from government and industry
- Multi-disciplinary teams, stakeholder engagement and participatory process involving patients and carers
- Evidence appraisal and transparent processes
20
Q
Hierarchy of evidence and health policy: Evidence-Based Policy (EBP)
- Health policy is developed based on
- Using the best quality healthcare … overall
- To ensure fair and equitable access to most effective and cost-effective healthcare
- Most research in medicine is produced by … and medical device companies – who want to later sell the product they make
- Cochrane is not-for-profit international organisation, which only collaborates with non-commercial organisations (…% of NICE and WHO guidelines use Cochrane Reviews)
- NICE guideline creation
- Independence from government and industry
- Multi-disciplinary teams, … engagement and participatory process involving patients and carers
- Evidence appraisal and … processes
A
- Health policy is developed based on
- Using the best quality healthcare evidence overall
- To ensure fair and equitable access to most effective and cost-effective healthcare
- Most research in medicine is produced by pharmaceutical and medical device companies – who want to later sell the product they make
- Cochrane is not-for-profit international organisation, which only collaborates with non-commercial organisations (80% of NICE and WHO guidelines use Cochrane Reviews)
- NICE guideline creation
- Independence from government and industry
- Multi-disciplinary teams, stakeholder engagement and participatory process involving patients and carers
- Evidence appraisal and transparent processes
21
Q
Hierarchy of evidence and health policy: Evidence-Based Policy (EBP)
- Factors that may inform decisions about the strength of recommendations for policy options:
- Uncertainty about
- Balance of … versus …, whether net … are worth costs/resource use
- F.. of the intervention
- Q.. of evidence
- Uncertainty or … in the values and preferences among stakeholders
- Or l… factors influencing the translation of evidence into practice
- Ease of implementation at the systems level
- Government, finance, implementation strategies and resources
- Socio-political correctness
- Relation to existing policies
- Socio-political values
- Economic considerations
- Uncertainty about
A
- Factors that may inform decisions about the strength of recommendations for policy options:
- Uncertainty about
- Balance of benefits versus harms, whether net benefits are worth costs/resource use
- Feasibility of the intervention
- Quality of evidence
- Uncertainty or variability in the values and preferences among stakeholders
- Or local factors influencing the translation of evidence into practice
- Ease of implementation at the systems level
- Government, finance, implementation strategies and resources
- Socio-political correctness
- Relation to existing policies
- Socio-political values
- Economic considerations
- Uncertainty about
22
Q
Hierarchy of evidence and health policy: Evidence-Based Policy (EBP)
- Factors that may inform decisions about the strength of recommendations for policy options:
- Uncertainty about
- Balance of benefits versus harms, whether net benefits are worth costs/resource use
- Feasibility of the …
- Quality of evidence
- Uncertainty or variability in the values and preferences among stakeholders
- Or local factors influencing the translation of evidence into practice
- Ease of … at the systems level
- Government, finance, implementation strategies and resources
- Socio-political correctness
- … to existing policies
- …-… values
- … considerations
- Uncertainty about
A
- Factors that may inform decisions about the strength of recommendations for policy options:
- Uncertainty about
- Balance of benefits versus harms, whether net benefits are worth costs/resource use
- Feasibility of the intervention
- Quality of evidence
- Uncertainty or variability in the values and preferences among stakeholders
- Or local factors influencing the translation of evidence into practice
- Ease of implementation at the systems level
- Government, finance, implementation strategies and resources
- Socio-political correctness
- Relation to existing policies
- Socio-political values
- Economic considerations
- Uncertainty about
23
Q
Hierarchy of evidence and health policy: Evidence-Based Policy (EBP)
- Factors that may inform decisions about the strength of recommendations for policy options:
- Uncertainty about
- Balance of benefits versus harms, whether net benefits are worth …/…. use
- Feasibility of the intervention
- Quality of evidence
- Uncertainty or variability in the values and preferences among …
- Or local factors influencing the translation of evidence into …
- Ease of implementation at the systems level
- Government, finance, implementation strategies and resources
- Socio-political correctness
- Relation to existing policies
- Socio-political values
- Economic considerations
- Uncertainty about
A
- Factors that may inform decisions about the strength of recommendations for policy options:
- Uncertainty about
- Balance of benefits versus harms, whether net benefits are worth costs/resource use
- Feasibility of the intervention
- Quality of evidence
- Uncertainty or variability in the values and preferences among stakeholders
- Or local factors influencing the translation of evidence into practice
- Ease of implementation at the systems level
- Government, finance, implementation strategies and resources
- Socio-political correctness
- Relation to existing policies
- Socio-political values
- Economic considerations
- Uncertainty about
24
Q
Hierarchy of evidence and health policy: Evidence-Based Policy (EBP)
- Factors that may inform decisions about the strength of recommendations for policy options:
- Uncertainty about
- … of benefits versus harms, whether net benefits are worth costs/resource use
- Feasibility of the intervention
- … of evidence
- Uncertainty or variability in the values and preferences among stakeholders
- Or local factors influencing the … of evidence into practice
- Ease of implementation at the … level
- Government, finance, implementation strategies and resources
- Socio-political correctness
- Relation to existing policies
- Socio-political values
- Economic considerations
- Uncertainty about
A
- Factors that may inform decisions about the strength of recommendations for policy options:
- Uncertainty about
- Balance of benefits versus harms, whether net benefits are worth costs/resource use
- Feasibility of the intervention
- Quality of evidence
- Uncertainty or variability in the values and preferences among stakeholders
- Or local factors influencing the translation of evidence into practice
- Ease of implementation at the systems level
- Government, finance, implementation strategies and resources
- Socio-political correctness
- Relation to existing policies
- Socio-political values
- Economic considerations
- Uncertainty about
25
_Asthma: The case of the not-very-EBP_
* *Expert Panel Report 3- National Asthma Education Program USA: Guidelines for Diagnosis and Management***
* 1) Recommendation to encase mattresses in allergen-impermeable covers
* Supporting references incl. an editorial, a study involving multiple allergens, and a study incl. some patients with no dust mite sensitivity. 5 trials showed no benefit for mattress encasings.
* Did not cite Cochrane meta-analysis of 49 RCTs (2,733 patients) showing no effectiveness.
* 2) Does not acknowledge that evidence-based recommendations for inhaled corticosteroids (ICS) in asthma apply exclusively to ...-...
* ... tend to be excluded from many trials seeking FDA approval for ICS in asthma Other trial evidence suggested no benefit for ICS in ...-...
* 3) Recommendations to prescribe daily ...-dose ICS for mild persistent asthma
* Despite only one trial showing effectiveness with respect to severe asthma-related events, and which also suggested side effects of growth suppression in ... (also shown in other studies).
* 1) Recommendation to encase mattresses in allergen-impermeable covers
* Supporting references incl. an editorial, a study involving multiple allergens, and a study incl. some patients with no dust mite sensitivity. 5 trials showed no benefit for mattress encasings.
* Did not cite Cochrane meta-analysis of 49 RCTs (2,733 patients) showing no effectiveness.
* 2) Does not acknowledge that evidence-based recommendations for inhaled corticosteroids (ICS) in asthma apply exclusively to **non-smokers**
* Smokers tend to be excluded from many trials seeking FDA approval for ICS in asthma Other trial evidence suggested no benefit for ICS in non-smokers
* 3) Recommendations to prescribe daily **low**-dose ICS for mild persistent asthma
* Despite only one trial showing effectiveness with respect to severe asthma-related events, and which also suggested side effects of growth suppression in **children** (also shown in other studies).
26
_Asthma: The case of a certain uncertainty_
* NICE NG80: Asthma: diagnosis, monitoring and chronic asthma management
* 1) November 2017
* Recommended ... the ICS dose as part of a self-management programme for children and young people with deteriorating asthma
* 2) February 2020
* Acknowledged the 2017 ICS ... dose recommendation was based on limited evidence mainly in adults and new evidence did not support it
* Although no evidence that ... dose was harmful (incl. on child growth), therefore recommendation to increase ICS removed but no recommendation was made not to increase ICS
* Therefore allowing ... decision-making for CYP who have varied and changing support needs
* NICE NG80: Asthma: diagnosis, monitoring and chronic asthma management
* 1) November 2017
* Recommended **quadrupling** the ICS dose as part of a self-management programme for children and young people with deteriorating asthma
* 2) February 2020
* Acknowledged the 2017 ICS **quadrupled** dose recommendation was based on limited evidence mainly in adults and new evidence did not support it
* Although no evidence that **quadrupled** dose was harmful (incl. on child growth), therefore recommendation to increase ICS removed but no recommendation was made not to increase ICS
* Therefore allowing **individualised** decision-making for CYP who have varied and changing support needs
27
_Allergy: The case of the cow milk protein allergy_
* Non-IgE mediated cow’s milk protein allergy in breastfed infants
* Premise:
* In an ...-... infant, if a mother ingests cow’s milk, the child will develop an allergy
* Either the mother needs to stop consuming dairy or the child should switch to a non-dairy formula
* 2006-2016
* NHS prescriptions for formula rose from 100,000 to over ... annually
* NHS spend from ...% from £8.1m to over £60m annually
* BUT: .....
* * * Guideline endorsement; NICE 2011, Milk Allergy in Primary (MAP) Care guideline 2013, International iMAP guidelines 2017
* Non-IgE mediated cow’s milk protein allergy in breastfed infants
* Premise:
* In an **exclusively-breastfed** infant, if a mother ingests cow’s milk, the child will develop an allergy
* Either the mother needs to stop consuming dairy or the child should switch to a non-dairy formula
* 2006-2016
* NHS prescriptions for formula rose from 100,000 to over **600,000** annually
* NHS spend from **700**% from £8.1m to over £60m annually
* Guideline endorsement; NICE 2011, Milk Allergy in Primary (MAP) Care guideline 2013, International iMAP guidelines 2017
* BUT!! see slide attached
28
_Allergy: The case of the cow milk protein allergy (2)_
* Many more ... allergy than ... allergy guidelines – formula industry support e.g. 5/11 authors NICE 2011
* Conflicts in medical education e.g. Allergy UK, British Society for Allergy and Clinical Immunology
* Broad impacts of ...-diagnosis
* individual child and parent health, social and psychological issues
* broader ... and economic costs
* mortality - near universal level of ... could prevent 823 000 annual child deaths \< 5 year
* Many more **milk** allergy than **food** allergy guidelines – formula industry support e.g. 5/11 authors NICE 2011
* Conflicts in medical education e.g. Allergy UK, British Society for Allergy and Clinical Immunology
* Broad impacts of **over**-diagnosis
* individual child and parent health, social and psychological issues
* broader **societal** and economic costs
* mortality - near universal level of **breastfeeding** could prevent 823 000 annual child deaths \< 5 years
29
_Allergy: The case of the cow milk protein allergy_
* Guidelines suggest ...-...% in non-breastfed infants
* No good epidemiological evidence in breastfed infants
* No indication of matching jump in prevalence
* Basic evidence regarding non-IgE mediated allergy when only drinking breast milk is very ... and quite ...
* Limited evidence of the ... onset allergy – should be easier to detect
* Just because breastfed, doesn’t mean ... came from the breast milk
* Non-specific and common symptoms
* Test requires e..., d... and re-introduction
* Encouragement of over-diagnosis in context of conflicts of interest
* Capitalising on parenting ...
* Guidelines suggest **1-3**% in non-breastfed infants
* No good epidemiological evidence in breastfed infants
* No indication of matching jump in prevalence
* Basic evidence regarding non-IgE mediated allergy when only drinking breast milk is very **limited and quite weak**
* Limited evidence of the **rapid** onset allergy – should be easier to detect
* Just because breastfed, doesn’t mean **allergen** came from the breast milk
* Non-specific and common symptoms
* Test requires **exclusion, delay** and re-introduction
* Encouragement of over-diagnosis in context of conflicts of interest
* Capitalising on parenting **attitudes**
30
_Broad impacts of over-diagnosis (cow milk protein allergy)_
* individual child and parent ..., social and ... issues
* broader ... and ... costs
* mortality - near universal level of breastfeeding could prevent ...,... annual child deaths \< 5 years
* individual child and parent **health**, social and **psychological** issues
* broader **societal** and **economic** costs
* mortality - near universal level of breastfeeding could prevent **823 000** annual child deaths \< 5 years
31
_Allergy: The case of the cow milk protein allergy_
_Milk Allergy in Primary (MAP) Care guideline 2013→International iMAP guidelines 2017_
* Simple and accessible algorithms for UK primary care clinicians
* Based on existing international consensus guidelines, not new evidence review
* In 2018, guidelines were criticised for
* Promoting ... of CMPA
* ... impacting breastfeeding
* ... impact on guidelines
* In 2019, guidelines were updated to
* Reflect apparent prevalence of CMPA at 1.28% CMPA, half non-... mediated
* Emphasise using ... judgement when interpreting symptoms
* Draw direct attention to danger of ...
* Emphasise need to ... cow’s milk
* Involve more ... input from individuals and organisations with no industry ties
* Simple and accessible algorithms for UK primary care clinicians
* Based on existing international consensus guidelines, not new evidence review
* In 2018, guidelines were criticised for
* Promoting **overdiagnosis** of CMPA
* **Negatively** impacting breastfeeding
* **Industry** impact on guidelines
* In 2019, guidelines were updated to
* Reflect apparent prevalence of CMPA at 1.28% CMPA, half non-**IgE** mediated
* Emphasise using **clinical** judgement when interpreting symptoms
* Draw direct attention to danger of **overdiagnosis**
* Emphasise need to **reintroduce** cow’s milk
* Involve more **stakeholder** input from individuals and organisations with no industry ties
32
_Allergy: The case of the cow milk protein allergy_
_Milk Allergy in Primary (MAP) Care guideline 2013→International iMAP guidelines 2017_
* Simple and accessible algorithms for UK primary care clinicians
* Based on existing international consensus guidelines, not new evidence review
* In 2018, guidelines were criticised for
* Promoting overdiagnosis of CMPA
* Negatively impacting breastfeeding
* Industry impact on guidelines
* In 2019, guidelines were updated to
* Reflect apparent prevalence of CMPA at ...% CMPA, half non-IgE mediated
* Emphasise using clinical judgement when interpreting ...
* Draw direct attention to danger of overdiagnosis
* Emphasise need to reintroduce cow’s milk
* Involve more stakeholder input from individuals and organisations with no ... ties
* Simple and accessible algorithms for UK primary care clinicians
* Based on existing international consensus guidelines, not new evidence review
* In 2018, guidelines were criticised for
* Promoting overdiagnosis of CMPA
* Negatively impacting breastfeeding
* Industry impact on guidelines
* In 2019, guidelines were updated to
* Reflect apparent prevalence of CMPA at **1.28**% CMPA, half non-IgE mediated
* Emphasise using clinical judgement when interpreting **symptoms**
* Draw direct attention to danger of overdiagnosis
* Emphasise need to reintroduce cow’s milk
* Involve more stakeholder input from individuals and organisations with no **industry** ties
33
_EBM and EBP: Hierarchy of evidence_
* Over-value of quantitative data/ analysis, ... significance
* Over-value of the idea of a single ‘...’
* Choice between competing sets of concerns, and not just technical evaluations of effectiveness, may be more relevant
* Multiplicity of evidence is best to support ... - all levels of investigations needed for full understanding of disease/phenomenon
* Over-value internal validity....(RCT, causal study design, bias appraisal)
* ...under-value external ... (local and individual context)
* Lack of consideration of importance of different outcomes and their primacy to patients e.g., morbidity, mortality, quality of life outcomes versus surrogate and other outcomes – hard-to-... outcomes
* Less evidence for ‘...-life’ patients e.g. with comorbidities, long-term conditions, conditions/interventions less suited to quantitative study/RCTs
* Under-value of ... evidence
* Under-value of ... perspectives, doctor expertise and judgement, tailoring and individualised medicine
* Over-value of quantitative data/ analysis, **statistical** significance
* Over-value of the idea of a single ‘**evidence**’
* Choice between competing sets of concerns, and not just technical evaluations of effectiveness, may be more relevant
* Multiplicity of evidence is best to support **causation** - all levels of investigations needed for full understanding of disease/phenomenon
* Over-value internal validity....(RCT, causal study design, bias appraisal)
* ...under-value external **validity** (local and individual context)
* Lack of consideration of importance of different outcomes and their primacy to patients e.g., morbidity, mortality, quality of life outcomes versus surrogate and other outcomes – hard-to-**measure** outcomes
* Less evidence for ‘**real**-life’ patients e.g. with comorbidities, long-term conditions, conditions/interventions less suited to quantitative study/RCTs
* Under-value of **mechanism** evidence
* Under-value of **patient** perspectives, doctor expertise and judgement, tailoring and individualised medicine
34
Nonetheless... (hierarchy of evidence importance)
35
... is the explicit process of searching, identifying, appraising and applying the best evidence in making decisions about (and with!) individual patients
**EBM is the explicit process of searching, identifying, appraising and applying the best evidence in making decisions about (and with!) individual patients**
36
The hierarchy of evidence is... (2)
* The hierarchy of evidence
* **Is a grading of what evidence we assess to have the most to the least risk of bias**
* **Is used to inform health policy (EBP)**
37
..., like many other organisations elsewhere, create/update policy based on reviewing and synthesising available evidence paying attention to the hierarchy of evidence – and usually using ... Reviews to inform guidelines
**NICE**, like many other organisations elsewhere, create/update policy based on reviewing and synthesising available evidence paying attention to the hierarchy of evidence – and usually using **Cochrane** Reviews to inform guidelines
38
Some criticisms of .../... and hierarchy of ... include the (over-) value placed on ... validity and (under-) value on ... validity
Some criticisms of **EBM**/**EBP** and hierarchy of **evidence** include the (over-) value placed on **internal** validity and (under-) value on **external** validity
