Research 500 Flashcards
What is Evidence informed practice
1) Best available reserach evidence
2) Professional/Clinical expertise
3) Client values & individualized needs
why evidence based practice is important
—> more effective tx/RMT
—> better supported by extended benefits insurance plans
—> more standardized among members of the profession
—> more reputable members of healthcare community
quantitative vs qualitative methodology
..
quantitative methods
Quantitative methods are based on the assumption that there is a uniform reality that can be observed, measured, and expressed in numbers
quantitative methods …
Also assumes linear cause and effect (i.e. a specific treatment causes a specific outcome)
Test hypotheses and use numbers to summarize information
Researchers manipulate the treatment setting and participants
—> They control as much of environment as possible
Major focus on ruling out “rival explanations” for outcomes
—> That is, accounting for confounding variables (more to come…)
—> Then use statistics to show probability of chance
—> If there is a low probability of chance, outcomes should be reproducable
qualitative methods
Qualitative methods assume that any observation effects [sic] what is being observed and vise versa
No assumption of a single reality
—> Any description is one of several realities that may be valid
No assumption of linear cause and effect
—> Multiple factors/causes may influence an outcome
Importance is placed on observing in the natural setting (i.e. there is no controlled setting)
Observer is part of the process
Data is collected via interviews, direct observation, and documents (such as journals, correspondence, and questionnaires/surveys)
quant vs qual?
Both have limitations, but both are useful parts of scientific research
Both assess the credibility of a study’s reliability and validity
Depending on the question being asked, some studies use both methods
—> Qualitative analysis often helps develop a specific and quantifiable hypothesis
—> Quantitative statistics can help illuminate patterns in qualitative studies
most healthcare research uses ___
However, most of health care research utilizes quantitative methods
Descriptive vs Explanatory Studies
..
descriptive study
Data used to describe a group/sample/population, with no intention of going beyond that sect, it is a descriptive study.
explanatory study
Studies that seek to make generalized statements about a population based on a studied sample are explanatory or inferential studies.
descriptive studies …
These studies do not test a theory, nor are they used to learn more about the sect.
These studies set the stage to eventually test a theory by first forming one.
documents and communicates clinician experiences, thoughts, or observations
—> USED TO FORM HYPOTHESIS
explanatory studies …
These studies look to see if, for example, a population benefits from an intervention.
Examines causes, etiology, or treatment efficacy by comparing groups
—> USED TO TEST HYPOTHESES
e.g. descriptive studies
case study
case series
correlational study
qualitative study
e.g. explanatory studies
(EXPERIMENTAL)
before & after treatment
clinical trial
(OBSERVATIONAL)
cross-sectional study
case-control study
cohort study
two categories of explanatory studies
EXPERIMENTAL studies
OBSERVATIONAL studies
Descriptive studies ____
FORM a hypothesis
Weaker evidence due to lack of control or comparison groups, but still explore cause and effect relationships
—> However, contribute to the weight of evidence when combined with consistent results from observational and experimental studies
Foundation for hypothesis based on observations
—> Provide detailed information that helps to refine the design of explanatory studies
Examples: case studies, case series, correlation studies, qualitative studies
Explanatory studies ____
TEST a hypothesis
Stronger evidence that clarifies or establishes cause and effect relationships
Provide evidence about research questions (i.e. disease prevalence or treatment efficacy)
Examples are divided in observational and experimental
—> Observational: cross-sectional, case-control, cohort studies
—> Experimental: before and after treatment studies, clinical trials
study designs pyramind – vs quality of evidence
HIGHEST QUALITY (top of pyramid)
1) Meta-analysis
2) Systmeic reviews
3) Critically Appraised literature & EBP guidelines
EXPERIMENTAL (explanatory)
4) RCT
5) non-RCT
6) cohort studies
7) case series/studies
8) individual case reports
9) background information, expert opinion, non-EBM guidelines
metanalysis
Studies focused on a particular question are grouped based on certain criteria
___
–> One or more databases are used to find all published articles meeting the criteria
—-> This can result in publication bias
___
–> Well defined criteria helps to reduce selection bias
Used to estimate size of treatment effect and/of settle any contradictory or inconclusive date
publication bias
“In published academic research, publication bias occurs when the outcome of an experiment or research study biases the decision to publish or otherwise distribute it. Publishing only results that show a significant finding disturbs the balance of findings in favor of positive results.”
Systematic Review
Similar to a meta-analysis, but includes non-published studies
___
–> This is important to note, as meta-analysis may have a publication bias, as studies with negative results often do not get published
___
–> This type of study helps to eliminate the publication bias
Best evidence synthesis: draws on a wide range of evidence and explores the impact of context, while also assessing validity, to determine inclusion
—>
For example, with LBP, when studies are too different from one another to be pooled, yet all address LBP, reviewers evaluate the selected studies by individually assessing each one for its validity
____ is a great resource for systematic reviews
Cochrane
RCT
Also known as randomized (controlled) trial, clinical trial, and/or intervention study
Provides most direct evidence of a cause-and-effect relationship following treatment
Powerful because study participants are randomly assigned to treatment or control group
treatment vs control group
“The treatment group (also called the experimental group) receives the treatment whose effect the researcher is interested in.”
“The control group receives either no treatment, a standard treatment whose effect is already known, or a placebo (a fake treatment to control for placebo effect).”
Cohort Study
Prospective, longitudinal, observational studies
—> Attempt to explain relationship between treatment and outcome
—> Prospective = outcome has not yet occurred
Members of the cohort are observed over a long period of time to see what the outcome is – e.g. some develop cardiovascular disease, others don’t, etc.
cohort define
Cohort = a group who all experience the same treatment (or other variable); or exhibit the same characteristics (e.g. risk factor for cardiovascular disease)
cohort study pros vs cons
Pros: provide strong observational “evidence” of a relationship between treatment/risk factors and the outcome
Cons: take a long time and are expensive
—> Attrition is often high (e.g. participants drop out, die, move out of the region, etc)
Before/After without Control
A type of case series, but rather than just observing an outcome, practitioner determines hypothesis; sets eligibility criteria and methods; collects baseline data; provides tx; measures outcome for a series of patients (making it an experimental case series)
—> Often performed by practitioners in their own practices
Before/After without Control risks
Risk of many weaknesses
Lacks a control group for comparison; potentially over-estimates the treatment effect
Data collection may be subjective and patients may over-report good outcomes (possibly because they already have a relationship with the practitioner)
May not be possible to generalize the findings to anyone outside the test group
Before/After with Control
Same, but less limitation d/t use of control, making it a stronger study
before after w/ control vs RCT
“The biggest difference between a controlled before-and-after study and a randomized controlled trial is a lack of randomization, which could introduce bias by not mitigating extraneous factors.”
case report
Describes events related to the care of a single patient
Better than anecdote, d/t thorough rationale with presentation, description, detail, and discussion along with directions for future investigation
Pros: can serve as the basis for a new hypothesis; can be used to report adverse reactions to treatment
Case series
Takes the case study a step further by combining individual case studies of similar patients
Pros: may be the first indication of a new phenomena [sic]
anecdote
A brief, revealing account of an individual
person or an incident
Not evidence, as there cause does not equal effect; lacks rationale, detail, and exploration
Can be used to create a case report/case
series
Correlation Study
Population survey using existing data about groups
NOT evidence, as correlation does not equal causation
It does not prove cause and effect. It describes an association between exposure and outcome.
Pros: quick way to see if there’s an association/correlation between an exposure and an outcome
Pros: relatively low cost study; simple and quick to conduct
correlation e.g.
“For example consider the relationship between the weather and ice cream sales. As temperatures rise ice cream sales increase. There is a correlation between the temperature and ice cream sales.”
“Correlation means two variables change together, while causation implies that one variable directly influences another. Correlation does not imply causation, meaning that just because two things are related doesn’t mean one causes the other.”
I.e. hot weather doesn’t directly cause someone to buy ice-cream
Components of a Research Article
abstract
introduction
methods
results/findings
conclusion/discussion
references
abstract
Summary including background, purpose, design, methods, results, conclusion, and discussion
introduction
Thorough description of the purpose/importance, states the research question, and includes a literature review (which places a study in context)
States the purpose for conducting the research
Introduces the research question which the study addresses (sometimes the hypothesis)
Methods
Detailed description of how the study was carried out
Readers should be able to decide if other explanations exist that explain the findings, or if the authors’ conclusions are strong
Method should be so exact that the study is replicable by other researchers
Results/Findings
Description of the analysis of the study data
Can be qualitative or quantitative
Objective; neither supports nor dismisses the hypothesis or research question
Conclusion/Discussion
Answers the question, “so what do these results mean in terms of the research question?”
Authors interpret the research results
As the discussion sometimes puts forward the authors’ informed opinions, it’s good when they cite other studies that point to similar results
References
Listing of other research, articles, etc that the authors consulted while preparing the article you’re reading
References should provide a good source of further reading
Hypothesis
statement that can be demonstrated to be true or false through the methodical gathering and analysis of empirical information or data.
—>
- An educated guess on how things work
- Should be testable and measurable
hypothesis generally has ___ & ___
1) independent variable - thing to be changed
2) dependent variable - thing to be measured
null hypothesis?
Thenull hypothesis is the commonly accepted fact; it is the opposite of the alternate hypothesis.
—> Researchers work to reject, nullify or disprove the null hypothesis by coming up with an alternate hypothesis that they believe explains a phenomenon and, therefore, rejects the null.
Technically, researches do not prove a hypothesis, but disprove a null.
null hypothesis e.g.
assumes there is no relationship b/w two variables
“Cats show no preference to food based on shape”
“plant growth is not affected by light colour”
“age has no effect on musical ability”
null hypothesis other e.g.
alternative hypothesis:
“Application of bio-fertilizer ‘x’ increases plant growth”
—>
alternative hypothesis is a hypothesis which researcher tries to prove
- it is denoted by H1
- opposite of null hypothesis (H0)
null hypothesis:
“Application of bio-fertilizer ‘x’ do not increase plant growth”
—>
null hypothesis is hypothesis which researcher tries to disprove, nullify
- it is denoted by H0
- opposite of alternative hypothesis (H1)
independent variable
Cause
Influencer
Manipulating
Dependent Variable
Effect
What is being influenced
Measuring
—>
Any measurable changes depend on the independent variable
Confounding variable
(AKA extraneous variable)
any variable other than the independent variable that influences the dependent variable… can/should be controlled or tested itself in a high quality study.
Types of Experiment Designs
By number of independent variables
By subject assignment
By number of independent variables
Simple (1 independent variable) vs complex (>1 independent variable)
By subject assignment
Between-subjects designs (independent designs): different subjects are used in each group
Within-subjects designs (repeated measures designs): the same subjects are used in each group
Mixed designs: include both between and within-subject components
between subject designs
each participant participates in one and only one condition of the experiment
within subject designs
all participants participate in all of the conditions of the experiment
Class #2
Statistics
Internal vs External Validity
Ethics and Peer Review
Critiquing an Article
Statistics
Statistics is the study of the collection, organization, analysis and interpretation of collected data
descriptive statistics
Descriptive statistics present the study’s results without generalizing them to a larger group
The purpose is to present data characteristics and summarize them as the average value for each variable, as well as the amount of variation of the data around the average
These include range, central tendency, and standard deviations (more to come soon)
Inferential statistics
Inferential statistics are used to draw conclusions about observed differences between groups and whether results can be extrapolated to larger populations (i.e. statistical significance)
Statistical significance tells us whether the results represent a meaningful
result or are likely due to chance. It also estimates the strength of
associations between variables
Values of p = < .05 are considered statistically signification
Tests used depend on the type of data involved
Analysis of variance (ANOVA)
Analysis of variance (ANOVA) measures both statistical significance and the strength of association
Examines the relationships among exposures/procedures/independent variables and how they affect outcome measures/dependent variables
Multiple analysis can be utilized when there are multiple outcomes measured (MANOVA)
analysis of covariance (ANCOVA)
If demographic variables are unequally distributed between groups in a study, an analysis of covariance (ANCOVA) can be used to make groups comparable
E.g. in a study assessing the effect of exercise on BP, there is a large attrition rate causing incomparable demographic groups (one much younger) by the end of the study. ANCOVA analysis would use age as a cofactor to eliminate any confounding effect on BP.
power analysis
Power analysis determines how many participants/subjects were needed to have 80% chance of acquiring statistical significance
This is typically done in the planning stages of a study, as authors prepare sample size and subject groups
There should be at least 10 subjects per outcome measure
Statistical significance provides ____
provides an estimate of how much of an outcome is due to the treatment(s) being measured.
I.e. confirms the outcome (e.g. decreased SNS) is most likely due to the treatment (e.g. Swedish massage)
As a research novice, the most important thing to note re: statistics in published articles is whether a study performed any tests of statistical significance and whether the authors provided rationale for the statistical tests they’ve used
Descriptive Statistics: Deep Dive
central tendency
mean
median
mode
dispersion
standard deviation
range
CENTRAL TENDENCY
of a set of data refers to a representation of the “middle” or the “expected” value of the data set; a central value for a probability distribution.
MEAN
the sum of all the values in a list divided by the number of people in the sample (aka the average)
MEDIAN
a number dividing the higher half of a list of values from the lower half. This is found by listing data numerically and finding the number in the exact middle.
If the data set is an even number (ie 28), take the mean one the two central numbers
MODE
the most frequent value assumed by a random variable.
The MODE is usually a different number than the median and mean, especially in a skewed distribution (see notes below)
Mode is not necessarily unique – ie bimodal or trimodal, etc.
DISPERSION
measures statistical variability and allows analysis of how widely dispersed the data are around the middle.
STANDARD DEVIATION
a measure of spread of numbers in a data set
If data points are close to the mean, then standard deviation is small. If many data points are far from the mean, then the deviation is large.
A low standard deviation (ie +/-1) means there is little variability
A high standard deviation means there is a lot of variability
Standard deviation may serve as a measure of uncertainty
If a datapoint falls in the tail ends of a standard deviation, it is an outlier and atypical.
RANGE
the different between highest and lowest values (subtract the two to deduce the range)
internal and external validity
..
internal validity = how well study is done
external validity = how applicable findings are in real world
External validity
is the ability to apply findings of research beyond the context of a study. That is, are the results generalizable to a larger group/population writ large?
E.g. can animal studies be replicated in humans? Was the study sample large and diverse enough to apply to the larger population?
Internal validity
is the capacity of a study to link cause and effect, within the study itself. A good study excludes other hypotheses as possible explanations for the findings (ie it rules out that the results are due to any other correlation and draws a direct and absolute connection between cause and effect).
I.e. correlation vs. causation
Threats to Internal Validity
aka confounding variables or sources of bias
Confounding variable
blurs or masks the effect of another variable (example: students’ reactions to coffee may be confounded by how much sleep they had the previous night.
Bias
formal “research” sense of bias is a systematic error (example: a tendency to underestimate placebo effect in treatment outcomes.
Selection
often an important threat to internal validity based on choosing specific or certain groups to survey/study. (example: asking Facebook users how technology has impacted their lives will have different results vs. asking non-computer-literate citizens how technology has impacted their lives. Therefore, you cannot make inferential claims about the effects of technology on society; just on the sample of Facebook users you questioned.)
When selecting samples, this threat can be limited by randomizing and including generalizable sample populations
Statistical Validity
Another potential threat to internal validity
Statistics can be misrepresentative/have errors
Type I error = a “false positive” – i.e. rejecting a null hypothesis that is actually true, meaning the research hypothesis is wrong (concluding a treatment works when it doesn’t)
Type II error = a “false negative” – i.e. failing to reject a null hypothesis that is actually false, meaning the research hypothesis is right (concluding a treatment doesn’t work when it does)
Another problem exists when the sample size is too small to make significant observations.
This relates to power: the more participants complete a questionnaire, and the fewer the measured outcomes, the greater the power.
If power is low (small sample, many treatment outcomes), a Type II error may occur.
E.g., a true treatment outcome may be occurring but cannot be seen in the data because the sample is too small, and/or the treatment outcomes are too many.
Reliability
Defined as:
- Dependability, consistency, and reproducibility of information obtained in a study
- The probability that the same results would be obtained with different subjects (i.e. generalizability)
Three common methods to test reliability
1) Test-retest: administering the same test/therapy/etc twice to the same group of subjects after a certain amount of time
2) Equivalent-forms method: two different, but equivalent, therapies to the same group at the same time
3) Internal-consistency method: the extent to which different researchers would be consistent in matching the same data and constructs
Validity vs. Reliability
Validity = accuracy;
reliability = consistency
Validity = degree to which an instrument measures what it supposed to measure;
reliability = degree to which an instrument measures the same way each time it is used under the same conditions
peer review process
scientists study something
scientists write about results
journal editor receives article & sends for peer review
peer reviewers read and provide feedback to editor
editor may send back to scientists who may revise & resubmit — may then be approved or rejected, or sent back
if article meets standards, it’s published in journal
ethics review
Ethics committees act to safeguard human rights and protect people who participate in health care research
According to the Panel on Research Ethics Government of Canada – “Ethical principals and guidelines play an important role in advancing the pursuit of knowledge while protecting and respecting research participants”
This is the ideal scenario; ethics violations have happened throughout the history of medical research
Ethics reviews follow three core principals:
1) Respect for persons: recognizes the intrinsic value of human beings and the respect and consideration that they are due
2) Concern for welfare: the quality of a person’s experiences of life in all its aspects
3) Justice: the obligation to treat fairly and equitably
