Research methods Flashcards
Briefly describe the Tuskegee Syphilis study in 1932-1972
Looked at effects of syphilis on untreated African-American men
Participants didn’t know they were in a study, most didn’t know they had syphilis
No consent given
Treatment was withheld even when penicillin was available.
When was the Nuremberg code made?
1947
What are the 10 points of the Nuremberg code?
1) Voluntary, well-informed consent of human subject
2) Experiment should aim at positive results for society that can’t be obtained any other way
3) Should be based on previous knowledge that justifies the experiment
4) Experiment should be set up in a way which avoids unnecessary physical or mental suffering or injuries
5) Shouldn’t be conducted when there is any reason that it implies a risk of death or disabling injury
6) Risks should be in proportion to & not exceed the expected humanitarian benefit
7) Preparation and facilities must be provided to protect subjects against risk
8) Staff conducting experiments must be fully trained and scientifically qualified
9) Human subjects must be free to immediately quit the experiment at any point
10) Staff must stop the experiment at any point when they observe that continuation would be harmful
Describe the Willowbrook hepatitis experiments
Experiments on children living in residential care with mental health problems or learning disabilities.
Intentionally infected children with hepatitis and observed progression
When was the declaration of Helsinki made?
1964
What is the Helskini declaration?
Ethical principles for research involving human participants
What are the basic principles of the declaration of Helsinki?
Respect for the individual, their right to self determination and the right to make informed decisions
Participant’s welfare must take precedence over interests of science and society
Ethical considerations must take precedence over laws and regulations
What studies require ethical approval?
Studies involving human / animal participants
Gathering novel data or information
Creating knowledge that can be generalised beyond the patient sample or setting
What projects don’t require ethical approval?
Service evaluations
Clinical audits
What ethics committee do you need to apply for if using NHS staff, patients or site?
NHS health research authority
What points need to be included for ethical approval?
Scientific rational Rigorous methodology Informed consent Vulnerability of participant groups Data protection, confidentiality and records management Research safety and wellbeing
What information must be included in the rigorous methodology for ethical approval?
Adherence to established methodology
Methods must be appropriate for producing new knowledge
Methods must be appropriate and feasible for proposed setting and sample
Study materials or equipment should be valid and reliable
Show skills for robust data management and analysis within the team
What methods are there for participant recruitment>
Face to face in clinic
Posters
Social media
What participant groups are considered vulnerable?
Children Patients with dementia Patients with learning difficulties People with mental health problems People in care facilities Over-researched populations
What is the definition of research?
Study which is trying to generate new knowledge about patients or conditions or healthcare services that will be generally true across a wider population than it is actually studying.
What is the definition of a service evaluation?
Evaluate or assess some part of a local healthcare service. Aims to improve service based on evidence collected.
Results from this project will only feed back into this particular service
What is a clinical audit?
Quality improvement process that seeks to improve patient care and outcomes through systematic review of care against explicit criteria and implementation of change.
What are the 5 stages of the audit cycle?
Identify Method Analyse Change Monitor
What does the identify stage in the audit cycle include?
Guidelines Evidence Patient and public involvement Process design Agree standards
What methods can be used for data collection of service evaluations or audits?
Patient notes / record review Survey of staff - paper / online Survey of patients - paper / online Telephone interview Face to face interview Focus groups
What are the problems with using patients notes to gather research data?
Limited type of information available
Need to ensure there are no patient identifies
What points need to be considered when creating a questionnaire?
What do we want to achieve? What information do we need to know to help this? Who should we ask and how should we ask them? Types of questions Choosing categories - standardised - analyse by age or scale - demographic factors Question writing
What is the format of questionnaire questions determined by?
What we want to know
What types of data we want to generate
What types of analysis we want to do
What are the 3 types of questions for a questionnaire?
Closed questions with yes / no or categorical answer
Closed statements answered on a scale
Open-ended questions
What are Likert scales?
Respondents specify level of agreement or disagreement on a symmetric scale for a series of statements
What can Likert scales be used for?
Knowledge Beliefs Attitudes Symptoms Traits
What assumptions do Likert scales assume?
Bipolar scaling method
Assumes distance between each response is equal
What should be considered when writing questions for Likert scales?
Is the question / statement loaded with social desirability?
Does the question require knowledge or information the respondent might not have?
Is the statement based on 2 premises?
Why do likert scales require many items?
Unlikely to identify a single item which represents the whole issue being measured
Reduces error of random responses or misunderstandings
Can design questionnaires to reduce skew
How are Likert scales analysed?
Add up scores across similar questions to make a scale (total score)
Report mean total score and standard deviation
What is a RCT?
Study in which participants are allocated randomly between an intervention e.g. treatment and a control group e.g. no treatment or standard treatment
Why are RCTs conducted?
Safety
- ascertain safe dose
- demonstrate safety and tolerability of new compound
- monitor adverse events profile
Efficacy / effectiveness
- demonstrate efficacy of new drug
- show the treatment T is superior or equivalent to treatment X
- demonstrate effectiveness and cost effectiveness
What is a confounder?
Variable that influences both the dependent variable and independent variable causing a spurious association
What does equipoise provide?
Ethical basis for medical research that involves assigning patients to the different treatment arms of a clinical trial
What is internal validity in an RCT?
Is the independent variable causing the dependent variable in the study?
What is external validity in an RCT?
To what extent can these findings be generalised to other people, situations and times
What is bias?
Any departure of results from the truth
How does bias occur in an RCT?
When systematic error is introduced into sampling
Testing by selecting
Encouraging one outcome or answer over others
What is the definition of bias in statistics?
A tendency of an estimate to deviate in one direction from a true value
What is selection bias in an RCT?
Systematic differences between baseline characteristics of groups that are compared
Study sample doesn’t represent target population
What is performance bias in an RCT?
Systematic differences between groups in the care that is provided or in exposure to factors other than the interventions of interest
What is attrition bias?
Systematic differences between groups in withdrawals from a study
What is observer / detection bias?
Systematic differences between groups in how outcomes are determined
Outcome measures don’t adequately capture outcome of interest.
What types of trials can be blinded?
Drug trials
Surgery vs non surgical
Psychological interventions
What is a single blinded RCT?
Patients are blinded
What is a double blinded RCT?
Blinding of patients and treating physicians
What is a triple blinded RCT?
Blinding of patients, treating physicians and study investigators
What is a P-value?
Probability that the difference observed could have occurred by chance if the groups compared were really alike
What type of study causes a high p-value?
Few observations with high variability
What type of study causes a low p-value?
Many observations with low variability
What is the clinically important effect?
Smallest difference in outcome between treatment groups that would demonstrate a clear advantage of 1 treatment over another in terms of chosen outcome measures
Why is it important to calculate sample size?
Too few participants - if you get a null hypothesis you don’t know if you have evidence of no effect or no evidence of an effect
Too many participants is unethical and expensive
What are parametric tests?
Make assumptions about the data, particularly that its normally distributed
Describe a right skew
Skewed part is to the right
Most cases are to the left
There is more count in the tail than expected in a normal distribution
Positive skew
Describe a left skew
Skewed part is to the left
Most cases are to the right
There is more count in the tail than expected in a normal distribution
Negative skew
What are the problems with a skewed distribution?
Mean is different from the median
The mean is highly influenced by the tail of the distribution
When is a T-test done?
Does one group have higher scores than another?
When is a chi-squared test done?
Does one group have a higher proportion of an outcome than another? (categorical)
What assumptions have to be made to carry out a T-test for an RCT?
Randomly sampled
Independent observations
Variances for each group are equal
Sample is large enough
When do you do an ANOVA test instead of a T-test?
If there are more than 2 groups
What are non-parametric tests?
Distribution free tests
Fewer assumptions but less power
What statistical tests are used for non-parametric tests?
Mann-Whitney U replace T-test
Kuskal Wallis replaces ANOVA
Spearmans Rho replaces correlation
When is the Mann-Whitney U / Wilcoxon rank sum test used?
To compare non-normally distributed continuous variables between groups if randomly sampled and independent observations
What are examples of qualitative data?
Patient voice in interviews Policy documents Historical records Open answers on questionnaires Viewpoints of clinician Media reports
What is qualitative research?
Multi-method in focus, involving an interpretive naturalistic approach to its subject matter. Qualitative researches study things in their natural settings, attempting to make sense of or interpret phenomena in terms of the meanings people bring to them
What are the benefits of qualitative research?
Depth and detail Openness in possible responses Avoid pre-judgements No need for restrictive hypothesis Good if we lack understanding of a concept and need more info
What are the limitations of qualitative research?
Results are highly subject
Not usually generalisable
Cant compare groups
Transcription and analysis can be time consuming
Rich data is hard to analyse on scale so need fewer participants
Complexities in data can be difficult to present
Describe the differences between qualitative and quantitative
Qualitative’s social theory is action whereas quantitative is structure
Qualitative methods involve observation and interview. Quantitative involve experimental survey.
Qualitative reasoning is inductive but quantitative is deductive
Qualitative questions are what is X? Quantitative is how many X’s?
Sampling method is theoretical for qualitative and statistical for quantitative
What are the reasons to use qualitative research?
Takes an interpretative, naturalistic approach to its subject matter Sets a patient-centred agenda Challenges received wisdom Theory can emerge from the data Can inform quantitative studies
What methods can be used for qualitative research?
Observation
Textual analysis / analysis of documents
Focus groups
Interviews
What is the purpose of textual analysis for qualitative research?
Understanding how language is used
Understanding how stories or public opinion develop over time
Looking for biases or deviations from the truth
Understanding patients’ lived experience and how they make meaning from it
Describe a focus group
Typically 8-10 participants
Group discussion about specified topics
Often a series of focus groups is conducted
Led by moderator / facilitator
Tape-recorded and transcribed
Revelations of views can spark off other participants to reveal broader insights than are possible from individual interviews
What are the 3 ways of structuring an interview?
Structured
Semi-structured
Unstructured / in depth
What does the recruitment sample depend on for qualitative research?
Purpose of the research
Resources available
Time available
Access to population
What are the different types of sampling for qualitative research?
Extreme / deviant case
Homogenous
Typical case
Snowball
What are the ways to evaluate validity of qualitative research?
Triangulation
Respondent validation
Further interviews or focus groups
Subsequent analysis / independent assessment
What are the 6 steps for planning and carrying out a qualitative study?
1) Formulate research Q
2) Choose method
3) Select and recruit sample
4) Design interview schedule and data collection
5) Interpretation of data
6) Validity checks
What is prevalence?
Frequency of cases in a given population at a designated time
What are cross-sectional studies?
Survey or test individuals in a population individually to determine prevalence of disease
Can also measure exposure
What is point prevalence?
Prevalence at a moment of time
What is period prevalence used for?
Measure things that fluctuate e.g. hay fever symptoms
What are the strengths of cross-sectional studies?
Measure prevalence and thus disease burden in whole population and subpopulations
Compare prevalence in exposed and non exposed to risk factors
Quick and inexpensive
Can be used to initially explore a hypothesis prior to another type of study
What are the weaknesses of cross-sectional studies?
Not suitable for rare diseases
Not suitable for diseases of short duration
Can’t separate cause and effect as they are measured at same time
Can’t measure rate of new changes arising and any changes therein.
What are cohort studies?
Group of people is followed through the and the onset of a disease / health event is measured
What is incidence?
Number of instances of disease / illness onset in a given period in a defined population
What are the strengths of cohort studies?
Can calculate incidence and relative risk
Can offer some evidence of cause-effect relationship
Can identify more than 1 disease related to single exposure
Good when exposure is rare
Minimises selection and information bias
What are the weaknesses of cohort studies?
Potential for losses to follow up Often requires larger sample Less suitable for rare diseases Expensive If retrospective, data availability and quality may be poor
How is relative risk calculated?
Incidence of disease among exposed divided by indigence of disease among non-exposed
What does a RR of less than 1.0 mean?
Risk in exposed group is less than risk in non-exposed group
Exposure may be protective against disease
What does a RR of equal 1.0 mean?
Risk in exposed group = risk in non-exposed group
Exposure isn’t associated with disease
What does a RR of more than 1.0 mean?
Risk in exposed group is higher than risk in non-exposed group
Exposure may be a risk factor for the disease
What does a RR of 1.5 mean?
Risk of outcome is 50% higher in exposed than unexposed group
What does a RR of 0.8 mean?
Risk of outcome is 20% lower in exposed than unexposed group
What are case control studies?
Groups who differ at outset on disease status
2 groups of participants - 1 with condition (cases), 1 without condition (controls)
What are the strengths of case-control studies?
Can offer some evidence of cause-effect relationship
Can identify multiple exposures
Good when disease / outcome is rare
Minimises selection and information bias
Retrospective -cheaper and often shorter duration
What are the weaknesses of case-control studies?
Can’t calculate prevalence or incidence
Less suitable for rare exposures
Can be hard to ensure exposure occurred before onset
Retrospective data availability and quality may be poor
Suitable control group may be difficult to find
How is risk calculated?
Outcome of interest divided by total number of all possible outcomes
How is the odds calculated?
Outcome of interest divided by outcome not of interest
How is an odds ratio calculated?
Odds in exposed divided by odds in unexposed
What is relative risk?
Incidence of disease in exposed divided by incidence of disease in unexposed
Calculated using total population at risk of developing disease
What is RR and OR not the same?
Odds ratio will always overestimate RR
What are confidence intervals?
Describe the range of values with a given probability e.g. 95% that the true value of a variable is contained within that range
What is sensitivity?
Proportion of people who have the disease who correctly test positive
What is specificity?
Proportion of people without the disease who correctly test negative
What is a positive predictive value?
Likelihood patient with positive test actually has the disease
What is a negative predictive value?
Likelihood patient with negative test result doesn’t have the disease
What happens to PPV and NPV as prevalence increases?
PPV increases
NPV decreases
What happens to predictive values as a disease becomes more common?
There will be less false positives for every true positive
There will be more false negatives for every true positive
Due to underlying frequency of disease increasing in the given population
What are the types of scientific report?
Journal article Conference abstract Conference proceeding Monograph or book Database or dataset Algorithm or code
What is a good scientific question?
Clearly state Answerable Maps onto at least 1 study design Important Not already answered
What should the title of a scientific research question include?
State in brief the study design, population and question addressed
What should an abstract include?
Background Aim of study Methods Results Conclusion
What does IMRaD stand for?
Introduction
Methods
Results
Discussion
What should be included in the introduction section of a scientific report?
Why this research is important or necessary
1) Describe problem or situation that motivates research
2) Describe current state of research in field
3) Reveal a gap or problem in the field
4) Explain how the present research is a solution to that problem
Present any hypothesis at end
What should be included in the methods section of a scientific report?
Study design - population, sample, methods, procedure, equipment, analyses
Ethics statements
What should be included in the results section of a scientific report?
Description of study sample, response rate and participant characteristics
Presentation of findings
Tables and figures
Adjusted comparisons
What should be included in the discussion section of a scientific report?
Summary of findings Reflect on meaning of data Connect findings to previous research Strengths of study Specific limitations Suggest additional research Clinical and/or research and/or policy implications of study
What are some guidelines for method reporting?
STROBE CONSORT COREQ SQUIRE PRISMA
How should you describe the population in a scientific report?
Target population
Study population
What is a type I error?
Study finds an association between A and B where none really exists so null hypothesis is wrongly rejected
What is a type II error?
Study finds no association between A and B when there really is one so the null hypothesis is wrongly accepted
Why does selection bias occur?
Recruitment from a setting which doesn’t represent target population
Those who agree to take part differ from those who don’t
Those who drop out differ to those who stay in
Why is it important to report studies in a standardised way?
Failure to understand quality of medical evidence is an ethical issue
May fail to introduce an effective treatment
May disrupt delivery of a treatment that works
May create a market for ineffective or dangerous therapies
Describe the hierarchy of evidence from bottom up
Case reports / opinion papers and letters Animal trials Cross sectional studies Case control studies Cohort studies RCT Meta-analysis and systematic reviews
What is a review?
Generic term for any attempt to synthesise the results & conclusions of 2 or more publications on a given topic
What are the types of review?
Traditional literature reviews / narrative reviews
Systematic reviews / meta analysis
Describe narrative reviews
Usually based on narrative or commentary & are produced by a ‘content expert’
Unsystematic approaches to collecting & summarising evidence can lead to bias
- Bias arising from studies included in review
- Bias arising from studies not included in review
- Bias arising from way review is done
What is a systematic review?
Summary of medical literature that uses explicit & reproducible methods to systematically search, critically appraise & synthesise on a specific issue.
Synthesises the results of multiple primary studies related to each other by using strategies that reduce bias & random errors
What is the aim of systematic reviews?
Systematic eg in its identification of literature
Explicit eg in its statement of objectives, materials & methods
Reproducible eg in its methodology & conclusions
What are the key elements of a systematic review?
Well formulated question
Comprehensive database search
Unbiased selection & abstraction process
Critical appraisal of selected studies
Synthesis of data & interpretation of results (evidence synthesis)
What is PICO-ST?
Population Intervention Comparison Outcomes Setting Timing
What is SPIDER?
Sample Phenomenon of Interest Design Evaluation Research type
What are the limitations of systematic reviews?
May be no trials / evidence
If you apply all your criteria effectively & get <5 studies, this is fine
Trials may be of poor quality
Results may still be inconclusive
Practice doesn’t change just because you have evidence of effect / effectiveness
What is a meta analysis?
Statistical combination of ≥2 studies to produce single estimate of effect of the health care intervention under consideration
What is a meta-synthesis?
Group of methods used to synthesise qualitative (or mixed) data to produce summary result / re-analysis of previous studies / study findings
What are the strengths of meta-analysis?
Statistically pools results from a range of trials
Effect of an intervention is seen over a bigger sample size than possible in a single trial
Results of each study are weighted for sample size
Results from a range of settings can be compared
Can evaluate homogeneity of results ie variability of effect sizes
Can explore variability ie test moderators of effect size (eg study quality)
What are the weaknesses of meta-analysis?
Heterogeneity – studies which measure things differently are difficult to combine or combination becomes meaningless
Publication bias
Requires lot of effort & subject-area expertise
Mechanics of statistics may obscure theory
