Research Design PowerPoint Lecture, Dr. Wofford Flashcards
Predictive Correlational Study
Predictive correlational study: Predict a behavior or response based on the observed relationship between that behavior and other variables
- Helps to validate diagnostic/prognostic info (validation of new measurement tool)
- Useful in developing models that can serve as a basis for clinical decision making and to understand factors that impact success with an intervention
Two types of Single Factor Designs for Independant Groups
- Pretest-Postest control group design
- Posttest-only control group design
1 Group Designs: One-way repeated measures design over time (1)
(Quasi-experimental design)
Used with research questions which examine effects of a treatment on physiological or psychological variables and how these effects develop over time
Retrospective vs Prospective research
•Prospective research: measure variables through direct recording in the present.
- •Researcher follows subjects as they progress through the intervention
- •Able to identify factors that influence outcomes
- •More reliable than retrospective studies
Retrospective research: examination of data that have been collected in the past
- •Medical records, surveys, secondary data sets/databases
- •Must question the accuracy of the data since researcher has no control over how the data was collected or variables operationalized
difference between single factor and multifactor designs
Single Factor: 1 IV with any number of levels
Multi-Factor: >1 IV
Single Factor Design for Independed Groups: Pretest-posttest control group design
Pretest-posttest control group design
- •2+ groups that are formed by random assignment
- •One group is the control and the other receives an intervention
- •Could be >1 experimental group
- •Both groups are measured on a DV before and after the intervention
- •Only one IV (group)- debatable (Wofford thought it was a repeated measures design, not really a single factor design because of the type of stats that would have to be used. She disagrees with book, but put it here as book classified.
Multi-Factor Design for Independent Group: Factorial Design:
- 2+ IVs-subjects are randomly assigned to the two variables (2 different types of groups)
- Factors=variables (2-way factor design has two IVs)
- Each factor has levels (3 x 3)- multiply the numbers to know how many groups (ie: 9 groups). very common to our profession.
in 3x3 example, the first 3 is the amount of factors, second 3 is the number of levels per factor
One Group Designs: Time series design (2)
(Quasi-experimental design)
- Often used when studying community interventions or policy changes
- Multiple measurements before and after treatment to document patterns or trends of behavior
NOT VERY COMMON IN OUR PROFESSION
more common in policy change researcher or public health.
Prospective Research
Prospective research: measure variables through direct recording in the present.
- Researcher follows subjects as they progress through the intervention
- Able to identify factors that influence outcomes
- More reliable than retrospective studies
Correlation vs Prediction
Correlational study: Purpose: describe nature of existing relationships among variables
- Correlation: measure of the degree of association among variables
- Provides rationale for clinical decisions or generation of hypotheses (often used to create hypothesese to drive additional studies)
•Predictive correlational study: Predict a behavior or response based on the observed relationship between that behavior and other variables
- •Helps to validate diagnostic/prognostic info (validation of new measurement tool)
- •Useful in developing models that can serve as a basis for clinical decision making and to understand factors that impact success with an intervention
Positive Likelihood Ratio
- Positive likelihood ratio (LR+)= true (+) rate/false (+) rate or sensitivity/1-specificity= indicates how many more times likely a (+) test will be seen in those with the target condition than those without the condition
- LR+>5 represent relatively important effects
Repeated Measures Designs: Multi-factor design
- Two-way design with two repeated measures
- >1 IV (usually time + group)
VERY COMMON
Two Types of Quasi-Experimental Designs
- One-Group Designs
- Multigroup Designs
what design is considered the scientific standard in clinical research for establishing a cause and effect relationship?
Pretest-Posttest Control Group Design
IV has two levels
(it can be configured in different ways)
it is the basic structiore of of an RCT
Posttest Probability
- •Posttest probability: Likelihood that a patient has the target condition after formal testing is conducted
- •A good test should have high posttest probability confirming the diagnosis or low posttest probability causing us to abandon the target condition
Longitudinal Research
Longitudinal study: researcher follows a cohort of subjects over time
- Perform repeated measures at different time intervals
- Able to describe patterns of change
- ie: Framingham heart study
- Threats to internal validity include testing effects (because of repeated measures), attrition, development of confounding variables
Example: Framingham Heart Study
Questionnaires
Questionnaire: structured self-administered surveys using pen & paper or e-formats
- Generally more efficient than interviews
- Provide anonymity
- Primary disadvantage is that there is a potential for misunderstanding questions or response choices
- Low response rate (34% normal)
Predictive Value Estimates
- •A clinical instrument must demonstrate that it is an efficient use of time and resources and that it yields a sufficient number of accurate responses to be clinically meaningful
- •Can be
- Positive Predictive Value or
- Negative Predictive Value
- •Can be
Interviews vs Questionnaires
Interview: researcher asks subjects specific Qs & records answers for later analysis
- Face-face or via telephone
- Major disadvantage is that they are costly and time intensive
•Questionnaire: structured self-administered surveys using pen & paper or e-formats
- Generally more efficient than interviews
- Provide anonymity
- Primary disadvantage is that there is a potential for misunderstanding questions or response choices
- Low response rate (34% normal)
Longitudinal vs Cross-sectional Research
Longitudinal study: researcher follows a cohort of subjects over time
- •Perform repeated measures at different time intervals
- •Able to describe patterns of change
- •ie: Framingham heart study
- •Threats to internal validity include testing effects (because of repeated measures), attrition, development of confounding variables
•Cross-sectional study: researcher studies a cohort of subjects at one point in time and draws conclusions about a population
- •More efficient than longitudinal
Pretest Probabability
Pretest probability: Likelihood that a patient has the target condition before any formal testing is conducted
Three types of EXPERIMENTAL Designs
- •Single factor design for independent groups
- •Multi-factor design for independent groups
- •Repeated measures designs
Multi-Factor Design for Independent Groups: Nested Design (2)
- Commonly used in educational studies
One Group Designs: One-Group pretest-postest design (2)
(Quasi-experimental design)
- One group of subjects receive an intervention and one set of repeated measures is taken before and after treatment
- IV is time with 2 levels (pre and post)
Repeated Measures Designs: 2 types of Single factor designs
- One-way repeated measures design
- Crossover design
Sensitivity vs Specificity
- Sensitivity: ability to obtain positive test when the target condition is really present
- True positive rate
- Increased sensitivity= fewer people with the disorder are missed
- 1-sensitivity= false negative rate
- Snout
- Specificity: ability to obtain negative test when target condition is really absent
- True negative rate
- Increased specificity= fewer people will have false positives
- 1-specificity= false positive rate
- Spin
Three types of One Group Designs (Quasi-Experimental Designs)
- One-group pretest-posttest design
- One-way repeated measures design over time
- Time series design
Case Control Study
Case control study: Groups of individuals are selected on the basis of whether or not they have the disorder/pathology of interest.
- Cases: Have the disorder
- Controls: Don’t have the disorder and serve as a comparsion group
- Advantageous because the samples are relatively easy to obtain and are useful for studying rare disorders
Descriptive Surveys (3 things)
- •Surveys are commonly used to collect information about a specific group
- •Generally focused on a group’s behaviors or attitudes
- •Overall purpose of the descriptive surveys are to provide an overall picture of the group’s characteristics
Systematic Review vs Meta-Analysis
Systematic review: rigorous process of searching, appraising, and summarizing existing information on a selected topic.
- Most commonly focused on the effectiveness of interventions
Meta-analysis: process which combines studies using a quantitative index to develop a single overall estimate of the intervention effect.
- Provide important results when several smaller studies are not sufficient to demonstrate meaningful outcomes
what article should we review to understand these concepts?
the Prone Lochman’s review article
Sensitivity And specificty (2x2 prediction table)
2x2 prediction table
Systematic Review
Systematic review: rigorous process of searching, appraising, and summarizing existing information on a selected topic.
- Most commonly focused on the effectiveness of interventions
Sensitivity
Sensitivity: ability to obtain positive test when the target condition is really present
- True positive rate
- Increased sensitivity= fewer people with the disorder are missed
- 1-sensitivity= false negative rate
- Snout
Liklihood Ratios (4 things)
- Likelihood ratio: How much more likely is it that a person has the diagnosis after the test has been performed
- Advantageous over sensitivity, specificity and predictive values because it is independent of disease prevalence and can be applied to multiple settings and patients
- Helps us to determine the posttest probability
- Can be Positive or Negative
Meta-Analysis
Meta-analysis: process which combines studies using a quantitative index to develop a single overall estimate of the intervention effect.
- Provide important results when several smaller studies are not sufficient to demonstrate meaningful outcomes
A single factor design is also called ____________.
one-way design
(and it is for investigations of one IV)
(it may include one or more DVs)
Developmental Research (2)
- •Developmental research involves description of developmental change and sequencing of behaviors in people over time.
- •Can be longitudinal or cross-sectional
Multi-factor Designs have _______ IVs
multiple (usually 2-3 IV)
Qualitative Research: Two methods of data collection
- Observation
- Interviews
DV
Dependant Variable
Five Typs of Descriptive Research
- •Developmental research
- •Normative research
- •Qualitative research
- •Descriptive surveys
- •Case study
Cross-Sectional Research
Cross-sectional study: researcher studies a cohort of subjects at one point in time and draws conclusions about a population
- More efficient than longitudinal
Repeated Measures Designs: Single factor design; One way repeated measures design
- •One group of subjects is exposed to all levels of one IV (a group of subjects is exposed to three types of interventions or conditions)
- •Order effects- randomize order of intervention/condition in order to decrease potential bias of testing all subjects in the same order (fatigue, learning or carryover effects)
RETROSPECTIVE RESEARCH
•Retrospective research: examination of data that have been collected in the past
- •Medical records, surveys, secondary data sets/databases
- •Must question the accuracy of the data since researcher has no control over how the data was collected or variables operationalized
Repeated Measures Designs: Single factor design; Cross-over design (3)
- Used when only 2 levels of an IV are repeated
- ½ subjects receive Treatment A followed by Treatment B and the other ½ receive Treatment B followed by Treatment A
- Washout period (big thing!)
Good for when you don’t have lots of subjects. Same people act as their own control group.
Case Studies
- •Case study/report: in-depth description of individual’s condition/treatment response
- •Case series: expansion ofcase study involving observ of several similar cases
- •Advantageous because it provides an understanding of the totality of an individual’s condition and care
- •Purpose:
- •Understanding unusual patient conditions
- •Provide example of innovative therapies
- •Generating and testing theory
- •Provide information for future research projects
Multi-Factor Design for Independent Groups: Randomized Block Design (2)
- •Is used when an extraneous variable is present which may influence differences between groups (ie: Gender). It is called a blocking variable. Gender doesn’t always influence differences between groups, but you can use this when id does.
- •The extraneous variable (blocking variable) becomes another IV
ONLY USED SOMETIMES IN OUR PROFESSION
Pretest vs Posttest Probability
- •Pretest probability: Likelihood that a patient has the target condition before any formal testing is conducted
- •Posttest probability: Likelihood that a patient has the target condition after formal testing is conducted
- •A good test should have high posttest probability confirming the diagnosis or low posttest probability causing us to abandon the target condition
READ THIS PART IN BOOK
do we want confidence interval to be narrow or wide?
the narrower the better
Two types of Multi Group designs
(quasi-experimental)
- •Nonequivalent pretest-posttest control group design
- •Similar to pretest-posttest experimental design, except subjects are not allocated randomly to groups
- •Nonequivalent posttest-only control group design
- •Similar to posttest-only experimental design, except subjects are not allocated randomly to groups
Cohort Studies
- Cohort: group of individuals who are followed together over time
- Cohort study: researcher selects a group of subjects who do not have the outcome of interest and study them to see if they develop the disorder
- May be purely descriptive or analytic (identifying risk associated with exposures)
- Advantageous over a case-control study because of its ability to determine the onset of the condition
- Cannot be used with rare disorders because too many subjects would have to be followed to document sufficient number of cases for analysis
Examples: Framingham study & Nurses study
IV
Independent Variable
Specificity
Specificity: ability to obtain negative test when target condition is really absent
- True negative rate
- Increased specificity= fewer people will have false positives
- 1-specificity= false positive rate
- Spin (acronym)
CV
covariate
Do we want Negative Likelihood ratio to be really high or really low (and what is considered acceptable and really really good)?
We want it low and
LR- < 0.2 is considered really really good
She didn’t say what was acceptable
Do we want Positive Likelihood ratio to be really high or really low (and what is considered sufficient and really really good)?
We want it to be high
LR+ > 5 represents sufficient
LR+ > 10 is considered really really good
,
Single Factor Design for Independent Groups: Posttest-only control group design
Posttest-only control group design (main difference is that there is not pre-test)
- •2+ groups that are formed by random assignment
- •One group is the control and the other receives an intervention
- •DV is only measured after the intervention; no pretest
- •ie: assessing hospital cost and length of stay- DV can only be assessed after intervention
- •Only one IV (group) (truly a single factor design), Def not a repeated measures design
Independant groups
Different subjects in each group
Confidence Interval: How narrow is narrow?
It’s a bit of a judgement call
Three Multi-Factor Factor Designs for Independent Groups
- Factoral Design (we use A LOT in our profession)
- Randomized Block design (used some in our profession)
- Nested Design (almost never used in our profession, don’t worry too much about)
Exploratory Research: Five types of Observational Designs
- •Retrospective and prospective research
- •Longitudinal and cross-sectional research
- •Correlational and prediction research
- •Case-control studies
- •Cohort studies
(these have some overlap)
Single-Subject Designs
- Involves systematic collection of repeated measures of a behavioral response over time. Aim is to observe trends & patterns & to evaluate variability of behav response over time. Looking more for trends in the graph more than stats
- Typically plotted on line graphs to observe trends in the data
- A= baseline time period; B= intervention time period
- Designs:
- •A-B design= one baseline time period and one intervention time period
- •A-B-A design= replicates one baseline time period following intervention
- •A-B-A-B design= begins with initial baseline data and ends with an intervention phase (strongest of the three)
- •Strongest design as it provides (2) opportunities to evaluate effect of an intervention
Seven Research Design Types
- Experimental
- Quasi-experimental
- Single-subject
- Observational
- Descriptive
- Interviews and Questionnaires
- Systematic reviews and Meta-analysis
Negative Liklihood Ratio
- Negative likelihood ratio (LR-)= false (-) rate/true (-) rate or 1-sensitivity/specificity= indicates how many more times more likely a (-) test will be seen in those with the target disorder than those without the disorder
- LR-
- We want negative liklihood ratio to be low
Acronym for Sensitivity
SnOut
Sn (sensitivity)
Out (rule out)
high sensitivity means it can be ruled out if not positive
Correlational Study
Correlational study: Purpose: describe nature of existing relationships among variables
- Correlation: measure of the degree of association among variables
- Provides rationale for clinical decisions or generation of hypotheses (often used to create hypothesese to drive additional studies)
Two Types of Repeated Measures Designs
- Single Factor Design
- Multi-Factor Design
Six questions to ask that will help determine what type of experimental design is appropriate (assuming you already know that experimental is appropriate)
- How many IVs are being tested?
- How many levels does each IV have, and are these levels experimental or control conditions?
- How many groups of subjects are being tests?
- How will subjects be assigned to groups?
- How often will observations of responses be made?
- what is the temoral sequence of interventions and measruements?
Qualitative Research
- •Qualitative research seeks to describe the complex nature of humans and how people perceive their own experiences within a specific social context
- •Uses subjects’ own words and narrative summaries as data versus numerical data
- •Methods of data collection:
- •Observation
- •Interviews
Positive Predictive Value
•Positive predictive value (PV+): estimates the likelihood that a person who tests positive actually has the disease
- •a/a+b
- •High PV+= strong estimate of the actual number of patients who have the target condition
Negative Predictive Value
•Negative predictive value (PV-): estimates the likelihood that a person who tests negative actually does not have the disease
- •d/c+d
- •High PV-= strong estimate of the actual number of patients who do not have the target condition
Interviews
Interview: researcher asks subjects specific Qs & records answers for later analysis
- Face-face or via telephone
- Major disadvantage is that they are costly and time intensive
Dependent groups
Same subject in each group
resulting in a sensitivity of confidence intervals
0.70 (95% CI: 0.49, 0.84)
95% sure that the sensitivity for this population falls between this range (0,49, 0.84)
I THINK
NORMATIVE STUDIES
- •Purpose of normative research is to describe typical values for characteristics of a given population
- •Norms are generally expressed as an average within a range of acceptable values
What is the most common design that Dr. Davies does and our Dept does?
Repeated Measures Design, Multi-Factor Design
Acronym for Specificity
SpIn (acronym)
Sp (specificity)
In (ruled in)
High specificity means it is ruled in
