RESS Curriculum Flashcards
Define statistical significance
The likelihood that the results obtained in a study were not due to chance alone (p values and confidence intervals may be used to assess statistical significance).
Define a p value
The probability that the observed difference between two treatment groups might have occurred by chance.
What does a p value <0.05 indicate?
If the p value <0.05 then the observed difference between the groups is so unlikely to have occurred by chance that we reject the null hypothesis (that there is no difference) and accept the alternative hypothesis that there is a real difference between the treatment groups.
Define a heuristic
Any approach to problem solving that employs a practical method that is not guaranteed to be optimal, perfect, or rational, but is nevertheless sufficient for reaching an immediate, short-term goal or approximation
What is the hierarchy of evidence?
A heuristic used to rank the relative strength of results obtained from scientific research.
Down side of hierarchy of evidence?
this focuses exclusively on ‘effectiveness’ and rarely pays attention to the quality of the studies
What is the pyramid shape used to illustrate in the hierarchy of evidence?
Increasing risk of bias
What is a case control study?
Selects participants on the basis of their outcome and works back to their exposure
E.g. patients who have developed a disease are identified and their past aetiological factors is compared with that of controls who do not have the disease.
In a case control study, the controls are used to estimate the exposure distribution (i.e. the proportion having the exposure) in the population from which the cases arose.
How is the odds ratio then calculated?
The exposure distribution in cases is then compared to the exposure distribution in the controls in order to compute the odds ratio as a measure of association
Advantage of case control studies?
Cheap
Main disadvantage of case control studies?
Prone to selection bias
Why are case control studies prone to selection bias?
Selection bias occurs when subjects for the ‘control’ group are not truly representative of the population that produced the cases.
May occur when ‘cases or controls are included/excluded from a study because of some characteristic they exhibit which is related to exposure to the risk factor under evaluation.
When is selection bias in case control studies a particular problem?
Where cases and controls are recruited exclusively from hospital/clinics → hospital patients tend to have different characteristics than the population e.g. higher levels of alcohol/smoking
What is a cohort study
Longitudinal study; an exposure is assessed (common characteristic) and then participants are followed prospectively to observe whether they develop the outcome
- Selects participants before outcome
- No control over outcome
What type of study is one that looks at the potential long-term consequences of the use of oral contraceptives?
Cohort study
What is a prospective study
Studies are planned in advance and carried out over a future period of time
What is a retrospective study
study looks at data that already exists and try to identify risk factors for particular conditions
Which type of study is used to try and identify risk factors for particular conditions?
Retrospective studies
What is a randomised controlled trial?
A number of similar people are randomly assigned to 2 (or more) groups to treatment groups (e.g. drug).
- One group has the intervention being tested (the experimental group)
- The other has an alternative intervention (placebo) or no intervention at all (the control group)
- The groups are followed up to see how effective the experimental intervention was
What can RCTs control?
Used to control factors not under direct experimental control
Which study type is regarded as the gold standard trial for evaluating the effectiveness of interventions?
Randomised controlled trials → is the most scientifically rigorous method of hypothesis testing available
Disadvantage of RCTs?
Expensive
Why is the RCT considered to provide the most reliable evidence on the effectiveness of intervention?
Because the processes used during an RCT minimise the risk of confounding factors influencing the results
The findings are likely to be closer to the true effect than the findings generated by other research methods.
What is meta analysis?
A method often used in systematic reviews to combine results from several studies of the same test, treatment or other intervention to estimate the overall effect of the treatment (a formal study of existing evidence).
Define research
Generates new knowledge where there is no (or limited) research evidence available and which has to potential to be generalizable or transferable
Define an audit
A quality improvement process that seeks to improve patient care and outcomes
What is the aim of an audit?
Quality improvement (patient care and outcomes)
How does an audit seek to improve quality of care?
Systematic review of care against explicit criteria and the implementations of change → Existing practice is measured against evidence-based clinical standards (explicit criteria).
What is a service evaluation?
Evaluates a current service or proposed practice to assess how well a service is achieving its intended aims.
What is the sole purpose of a ervice evaluation?
sole purpose of defining or judging the current service
Audit vs service evaluation?
A clinical audit is designed to answer the question → ‘Does this service reach a certain standard?’
A service evaluation is designed to answer the question → ‘What standard does/might this service achieve?’
What is an audit-cum-service evaluation?
Many studies conduct both an audit (to establish the extent to which clinical practice is achieving a particular standard) as well as a service evaluation (to establish what factors might be associated with those contexts)
Example → NICE guidelines recommend swallowing therapy “at least three times a week for as long as they continue to make functional gains”
Study 1 AUDIT: What proportion of patients with dysphagia are given swallowing therapy at least 3 times a week? (the proportion of patients who receive the care that meets that recommended by the standard/guideline)
Study 2 SERVICE EVALUATION: How many times a week are patients given swallowing therapy? (primarily interested in variation in the characteristics of healthcare delivery)
Study 3 AUDIT-CUM-SERVICE EVALUATION: What proportion of patients with dysphagia are given swallowing therapy at least 3 times a week in Consultant-led and non-Consultant-led clinics? (interested in whether variation in patient characteristics or the characteristics of healthcare delivery might influence whether a patient receives standard/recommended care)
What axis is the exposure on?
Independent → x axis
What axis is the outcome on?
Dependent → y axis
If the confidence interval around the odds ratio does include 1, what does this mean?
If the confidence interval includes or crosses 1, then there is insufficient evidence to conclude that the groups are statistically significantly different → cannot be confident there is an effect
Ethics vs governance?
Ethics → what is morally correct
Governance → what has received permission
What does ethical research aim to ensure?
Ethical research aims to ensure that the potential benefits of any given study never take precedence over the wellbeing (most studies impose burdens and risks) and rights (most studies require informed consent) of study subjects/participants.
What is he role of ethics & governance approval?
aim to ensure that no human (and many animal-based) studies take place without ensuring appropriate measures are in place
When is ethics approval particularly important?
when the study’s participants are unable to consent e.g. dead; unconscious; unwell; lack capacity
Give some examples where formal ethical approval and consent may NOT be needed
The changes are routine and not part of the study’s design and either:
a) all the data used has been collected during/as an established part of a routine practice OR;
b) the collection of any additional data does not impose excessive burdens or risks (though consent should be obtained)
What can you change without seeking further approval (a so-called ‘amendment’) from the Research Ethics Committee?
a) Add an additional invasive measurement to the data you want to collect
b) Change the content of the participant information leaflet
c) Change your original hypothesis
d) Recruit additional participants
Change your original hypothesis
What is a ‘conflict of interest’ in a study?
This refers to the situations in which financial or other personal considerations may compromise (or have the appearance of compromising) an investigator’s professional judgment in conducting or reporting research.
I.e. The responsibilities of a medical researcher may be at risk of being unduly influenced by a 2ary interest e.g. a financial gain or career advancement.
Define a public partnership
An umbrella term for describing activities which involve members of the public.
Define public engagement with research
Where information about research is provided and disseminated
- Dissemination events
- Social media
- Newsletters
- Websites & blogs
- Creative methods
Define public involvement with research
Where members of the public are actively engaged in shaping and developing research projects
Define public participation with research
Where people take part in a research study
Define ‘coproduction’ of a research project
Co-producing a research project is an approach in which researchers, practitioners and the public work together, sharing power and responsibility from the start to the end of the project:
- Redressing power differentials
- Knowledge of equal importance
- Clear roles and real responsibilities
- Accessibility e.g. language
What is the role of NICE?
To improve outcomes for people using the NHS and other public health and social care services by:
- Producing evidence-based guidance and advice for health, public health and social care practitioners
- Developing quality standards and performance metrics for those providing and commissioning health, public health and social care services
- Providing a range of information services for commissioners, practitioners and managers across health and social care
- Reduce variation in the availability and quality of treatments and care
- Help resolve uncertainty about which medicines and treatments work best and which represent best value for money for the NHS
Clinical guidelines vs quality standards?
Clinical guidelines → evidence based
Quality standards → guideline based
What is a clinical guideline?
A comprehensive set of recommendations for a particular disease or condition
What is a quality standard?
A standard is a level of quality or attainment
What is the GDG (guideline developing group)?
The GDG is established by the NCC or the Internal Clinical Guidelines Programme, and is responsible for developing the clinical guideline.
During development of the clinical guideline, the GDG: agrees the ‘review questions’ about treating and managing the condition that will guide the search for evidence.
What is the key role of the GDG?
make judgements based on evidence
Define a descriptive study
Classifies, describes, compares and measures data
Describe an analytical study
Focuses on cause and effect
Define a confounding variable
A confounder is a variable that influences both the dependent variable and independent variable, causing a spurious association
Define a covariate
An independent variable that can influence the outcome of a given statistical trial but is not of direct interest
What are the 5 different types of variable?
- Exposure
- Outcome
- Mediator
- Confounder
- Competing exposure
Define a mediator
A variable that links the independent and dependent variable, whose existence explains the relationship between the two variables
Define a confounder
A variable that influences both the independent variable (exposure) and the dependent variable (outcome)
Define a competing exposure
An ancestor of the outcome that is not related with the exposure
3 list approach to classify covariates:
Define 1ary data collection
Data that has been generated by the researcher (e.g. surveys, interviews, experiments) and is specially designed for understanding and solving the research problem at hand
Define 2ary data collection
Data that was formerly collected for other purposes than that of the current research study at hand (e.g. existing data generated by healthcare providers, government institutions such as medical electronic health records, disease registries)
Define a prospective study
Record/measure variables during the study period, with the outcome subsequently measured e.g. cohort study
Major pros of prospective studies?
- Major strength of a prospective cohort study is the accuracy of data collection with regard to exposures, confounders, and outcomes and other covariates
- Fewer sources of bias (e.g. recall or ascertainment)
- Less chance of unmeasured confounding
Major cons of prospective studies?
- Time and resource intensive
- Expensive
- Tendency to collect data on more variables than you need or can use in your analyses (but not if you use a DAG)
- Participants can drop out during a study (bias)
- Usually infeasible for rare outcomes
Define a retrospective study
Record/measure the outcome and then look back to record/measure the exposure and covariates e.g. case control studies and studies using existing patient records
What is a DAG (directed acyclic graph)?
Displays assumptions about the relationship between variables
- Directed → Graph has a direction (topological ordering)
- Acyclic → A graph without cycles (a cycle is a complete circuit), so you will never visit the same node twice
Pros of using newly collected data over existing data?
Helps ensure that you record all the variables you need, using the best techniques available, applied consistently
Cons of using newly collected data over existing data?
- More expensive & time consuming
- Harden to recruit participants
- Usually constitutes ‘research’ – therefore requires ethical approval
Pros of using existing data over newly collected?
Less expensive & less time consuming
Cons of using existing data over newly collected?
- Limited choice as to which variables have been measured
- Limited involvement in the (in)consistency of measurement
- Limited involvement in the level of ‘missingness’ tolerated
Define a registry
A collection of information about individuals, usually focused around a specific diagnosis or condition.
Define a measurement error
The difference between the measured value of a variable and its true value.
Define response bias
A general term for a wide range of tendencies for participants to respond inaccurately or falsely to questions (i.e. what interviewer wants/gets)
Define prestige bias
A bias towards copying ‘prestigious’ individuals (i.e. what appears favourable)
Define recall bias
Participants are systematically more or less likely to recall and relate information on exposure depending on their outcome (prospective/retrospective)
Define a proxy
A variable that is not in itself directly relevant but serves in place of an unobservable or immeasurable variable. E.g. intended variable = true body fat percentage, proxy variable = BMI
Define a latent variable
A variable that cannot be observed but can be detected by their effects on variables that are observable.
Define a sample
A sample is a subset of your population by which you select to be participants in your study.
Define sampling
The process of selecting a subset of a population which will be a representation of the whole population.
Define a target population
The total, finite population of people/contexts (to whom we wish to apply our study’s results and from which any study sample is drawn).
What should sampling ensure?
Sampling should ensure that the sample is a true representative of the population.
Probability sampling is a sampling technique, in which the subjects of the population get an equal opportunity to be selected as a representative sample.
Non-probability sampling → the elements that make up the sample are selected by nonrandom methods
What is sampling bias?
This involves problems in your sampling, which reveals that your sample is not representative of your population.
Most important source of sampling bias?
Most important are those arising from your inclusion/exclusion criteria.
What is a stratified random sample?
- The whole population is divided into homogeneous strata or subgroups according to a demographic factor (e.g. gender, age group, religion, socio-economic level, education, or diagnosis etc.)
- Randomly sample from the target population within strata (particularly to ensure equal numbers by exposure and confounder status)
- Every member of the target population within each stratum has the same chance/probability of being sampled
- The numbers in each stratum can be oversampled to strengthen analysis
Pros of stratified random probability sampling?
- Representative of population
- Unequal sampling of strata improves power for rare strata
How can you achieve a random/probability sample?
We use a systematic approach designed to ensure that every person/context in the ‘target population’ has an equal chance/probability of being included in the ‘study sample’
Methods:
- Table random numbers
- Computer generated list of random numbers
- Lottery methods
What is the purpose of a sample size calculation?
to ensure that your study is large enough to have a good chance of detecting any effect you hope to see
What does it mean if the confidence interval around the odds radio does not include one?
confident that the effect is genuine
How can you improve the chances that the confidence interval doesn’t include 1?
increase the sample size (a bigger sample to estimate from will make the estimate more accurate with less variability).
In which evaluation study activities can it be helpful to consult clinical and non-clinical stakeholders?
- Design of the research
- Development of the grant application (pre-protocol work)
- Undertaking/management of the research
- Analysis of data
- Dissemination of research findings
