Exam 2 Flashcards
What experimental design is not a true experiment and why?
- A quasi-experimental design
- lack randomization
- lack comparison groups
What is the gold standard of true experimental design?
Randomized Control Trial
What occurs in a true experimental design?
- Subjects are randomly assigned to at least 2 comparison groups
- Experiment enables control over most threats to internal validity and provides the strongest evidence for causal relationships
What is a completely randomized design?
- Between subject design
* Subjects assigned to groups based on a randomization process
What is a randomized block design?
- Subjects classified according to an attribute (blocking variable)
- Then randomized to treatment groups
What is a repeated-measures design?
- Within-subjects design (everyone gets same interventions)
* Subject acts as own control
How many independent variables do single-factor designs have?
*One independent variable
How many independent variables do multi-factor designs have?
*Two or more independent variables
What is a single-factor design?
- One way design
- 1 independent variable is investigated
- 1 or more dependent variables
- Looking at how many independent variables there are, not how many dependent variables there are
What occurs with a RCT with 2 groups based on random assignment?
- Pretest-posttest control groups design
- Independent groups= treatment arms
- Testing pre and post treatment
- Changes in experimental group are attributable to the treatment
- Establishes cause and effect relationship
- Change in the experimental group is the post result minus the pre result to get the overall result. The change becomes the dependent variable
What occurs in a 2-group pretest posttest design?
- Comparison group recieves a second form of the intervention
- 2 experimental groups formed by random assignment
- Control group is not feasible or ethical
- Compares new treatment with “standard care”
What is a multi-group pretest posttest control group design?
- Multiple intervention groups
* Includes a control group
What are pre-test post-test designs strong in?
*Strong in internal validity
*
How can the initial equivalence of groups be established?
*By pretest scores (important for inferring causality)
How is selection bias controlled in pretest posttest designs?
*Controlled because of random assignments
What should effect groups equally in pretest posttest designs?
*History, maturation, testing, instrumentation
How is the analysis of pretest posttest designs often done?
- Often analyzed using change scores
- difference between posttest and pretest
- Also can use analysis of covariance (ANCOVA) to compare posttest scores
- using pretest scores as covariate
What is a posttest only control group design?
- Same as pre-test posttest control group design
- EXCEPT no pre-test
- Used when dependent variables can only be assessed following treatment
- e.g. length of stay in hospital (see example pg 199)
- Used when pretest is impractical or detrimental
- Is an experimental design involving randomization and comparison groups
- Strong internal validity
- Assumes groups are equivalent prior to treatment
- works best with large samples to increase probability of “equivalency”
What occurs in a multi-factor design for independent groups?
- single factor designs have 1 independent variable (with 1 or more levels) and do not account for interactions of severable variables
- Multi-factor designs have 2 or more independent variables
What occurs in a factorial design?
- Factorial design incorporates two or more independent variables, with subjects randomly assigned to various combinations of levels of the two variables
- Two-way (two-factor) design has 2 independent variables
- Three-way (three-factor) design has 3 independent variables
What is a survey?
- A series of questions
- interview
- questionnaire (written/electronic)
- Can be used in:
- experimental studies, exploratory studies, descriptive studies
What occurs in an interview?
- Ask questions and record answers
- Structured format
- Unstructured format
What occurs in a structured interview?
- Standard set of questions
- Same questions in same order to all subjects
- Same response choices
What occurs in an unstructured interview?
- Less formal
- Open ended
- Conversational
- Often used in qualitative studies
What occurs in questionnaires?
- Structured surveys
- Self-administered
- Computerized or pen/paper
- Efficient as completed on subject’s own time
- Reduced bias from interactions with an interviewer
- Disadvantage is the potential for misunderstanding or interpreting questions
- Mail, electronic distribution
- Low return rates (30-60%) limit external validity of results
What are data collected via interview or questionnaire based on?
- SELF REPORT!
- no direct observation by the researcher of subject’s behavior
- potential for bias or inaccuracy
- recall bias
What should be asked when making a survey design?
- Delineate the overall research question
- What are the objectives (guiding questions) of the study?
- These objectives focus the content of the questionnaire
- Outline of the questionnaire (relate to objectives)
- Review existing instruments
- Can they be adapted for my study?
- Write questions that address each of the objectives
What do you ask when you distribute your preliminary draft of survey to colleagues?
- Ask for feedback
- Revise
- Distribute again
- Helps establish content validity of the instrument
What size sample should you do a pilot test on?
- Small sample of 5-10 research subjects
- interview respondents for feedback
- revise
- retest
How do you select a sample for surveys?
- All PTs in Michigan
* Purchase mailing lists
How do you contact respondents?
- Cover letter
* Follow Up
What do scales of surveys provide?
*Provide rating of degree to which subject possesses a characteristic/attitude/value
What are the scales of surveys?
- Likert:
- Strongly agree
- Agree
- Neutral
- Disagree
- Strongly disagree
- Likert scales coded 1-5
- Calculate overall score by adding answers
- 1 item does not carry more weight than others
- *visual analog scale
- Place a mark on the 100 mm line
What is the Delphi Technique?
- Experts complete multiple e.g. 3 rounds of questionnaires
- Researcher reviews and distributes findings after each round
- Eventually come to consensus on an issue
- e.g. what should entry level knowledge be for a particular topic
How do you do an analysis of survey data?
- Code the data e.g. male 1, female 0;
- Fear of falling 1, no fear 0
- SA=5; A=4; N=3; D=2; SD=1
What does descriptive statistics do?
- Summarize responses
- Mean age; years of education etc.
- Categorical data
- Frequency/percentages: 25 males (33% of sample)
- 40% SA; 30% A; 20% Neutral etc
- Scores on a scale may be summed Ex. mean ABC=84%
What does an IRB do?
- Must approve survey research
- Protection from psychological harm
- confidentiality
- Informed consent must be provided by participants
What is the best way for clinicians to seek evidence for interventions and assessments?
- Systematic Reviews
- Cochrane Collaboration
What should a study question be?
- Question is very specific
* Well described purpose statement
What is a narrative review?
*There’s a broad question to be answered with search strategies and selection of articles not usually described. Appraisal is not always rigorous and the conclusions are usually descriptive/qualitative
What is a systematic review like?
*The question is focused and the search strategies/databases are often described in detail. The selection of articles are rigorous based on specific criteria. Appraisal is very rigorous and the conclusion maybe qualitative or quantivative (meta-analysis)
What are the selection criteria of a systematic review?
- “Subjects” of the review are the studies
- Specify inclusion/exclusion criteria
- Based on:
- Types of studies
- Types of participants
- Types of interventions
- Types of outcome measures
What is the search strategy for a systematic review?
- Select keywords
- Identify resources
- Databases
- Grey literature (unpublished studies)
- Publication bias
- Conduct the search and retrieve relevant papers
How do you evaluate quality of selected studies?
- Critical review
- Record on a form
- Evaluate quality of design and data analysis
What are the types of study bias?
- Selection bias
- Performance bias
- Attrition bias
- Detection bias
What is the Jadad Rating scale?
- 3 questions
- Was study randomized (1 point if yes)
- Was study described as double blind (1 pt yes)
- Was there a description of withdrawals and dropouts (1 pt yes)
How many items is the PEDro Scale?
*11 items
What is Data synthesis?
- Heterogeneity (dissimilarity) or homogeneity of the included studies (variability across studies)
- Composition of treatment groups (inclusion/exclusion criteria)
- Design of study (including length of follow up)
- Management of patients (treatments provided/Presence of complications)
What are the analysis/synthesis of findings?
- Overall conclusions based on quality of evidence obtained
* Often summarize findings in a table
What is a forest plot?
- Represents the overall result of the meta-analysis
- square is the outcome for that study [relative risk (RR)]; size of square relates to weighting of study based on sample size
- line represents confidence interval (CI) around the RR
- diamond is combined overall estimate of results [includes pooled point estimate (center of diamond) and CI (horizontal tips of diamond)]
Describe the confidence interval (CI) of a forest plot?
*If a CI of a result crosses the line of no effect, then either a significant difference does not exist b/w the treatment and the control or the sample size was too small to show an effect
What is meta-analysis homogeneity?
*results of each individual trial are mathematically compatible with the results of the other trials
What are heterogeneous studies?
- If the CIs do not overlap; heterogeneous studies
- no common treatment effect across the studies
What is a two-way factorial design?
- Incorporates 2 independent variables
- effect of intensity (vigorous/moderate) on exercise behavior
- Effect of location (home/community center) on exercise behavior
- 2x2 design means 2 independent variables and 2 levels of each independent variables (4 groups)
What are the two-way factorial design main effects?
- Is there an effect of moderate versus vigorous exercise
- Is there an effect of exercsing at home or in community
- This is examining main effect of each independent variable
What are two-way factorial design interactions?
- Can also examine interaction effects between 2 independent variables
- Effect of 1 variable varies at different levels of the second variable
- e.g. maybe moderate exercise is more effective in changing exercise behavior but only when performed at a community center
When is a randomized block design used?
- When there is concern that an extraneous factor such as gender might influence differences between groups
- build the variable into the design as an independent variable
How many independent groups are in a repeated measures design?
- Up to now considered 2 independent groups
- experimental and control groups created by random assignment or by blocking
What is a repeated measures design?
- One group of subjects is tested under all conditions
- Each subject acts as their own control
- aka within-subjects design
What are the advantages to a repeated measures design?
- Subject differences can be controlled
- Ex: differences between experimental and control groups are nullified bc no groups used
- Physiological and other factors remain constant throughout the experiment
- Subjects act as own controls provides most equivalent “comparison group” possible
What are the disadvantages of repeated measures design?
- Learning/practice effects when one person repeats measurements over and over
- Carryover effects when exposed to multiple treatment conditions
- Must allow enough time for dissipation of previous effects so there’s not cross-over
Why might repeated measures designs not be considered true experiments?
- Bc no randomized comparison groups
- However if they incorporate randomization of the order of repeated treatments/interventions then can be considered experiment
What is a one-way repeated measures design?
*One group of subjects is exposed to all levels of one independent variable
What is Order Effects?
*were responses dependent on which condition preceded which other condition?
What is the Solution to problem of order effects?
*Randomize order of conditions/interventions for each subject so there is no bias in choosing order of testing
What is a two-way design with two repeated measures?
- 2 repeated measures (= 2 independent variables)
- Type of lift (squat/stoop)
- Orthosis (yes/no)
- Each person exposed to 4 test conditions
- 2-way design (2x2 design)
What is a Mixed design for multi-factor repeated measure design?
- 2 independent variables
- Exercise is independent factor
- -2 independent groups (experimental; control)
- Time is repeated factor
- -3 time periods (pretest; time period 1; time period 2)
- 2-way design (2 x 3 design)
What do Quasi-experimental designs lack compared to experimental designs?
- Random assignment
- Comparison group
- Or both
What are quasi-experimental designs?
- May involve non-equivalent groups
- May be a reasonable alternative to RCT
- Conclusions drawn must take into account biases of the sample
What occurs in a one-group pretest-posttest design?
- Effect of treatment is determined by change in pre- and posttest scores
- Pretest—intervention—posttest
- Vulnerable to threats to internal validity because no control group
- History
- Maturation
- Testing
What is a one-way repeated measure design over time?
- Effect of treatment over time
- Pretest-intervention-posttest 1- posttest 2
- no control group so internal validity threatened
What is a Non-equivalent pretest-posttest control group design?
- Similar to pretest-posttest experimental design EXCEPT subjects not assigned to groups randomly e.g. volunteers self-select groups
- EXP: Pretest–Intervention–Posttest
- CON: pretest–no intervention–posttest
What is a non-equivalent posttest only control group design?
- Static group comparison
- EXP: intervention –Posttest
- CON: no intervention –Posttest
What is a single subject design?
- Draw conclusion on treatment effects based on 1 patient’s response
- Controlled experimental approach
- Independent variable is treatment
- Dependent variable is target behavior (outcome)
- Also called N of 1 study or Time series designs
What is the Structure of a single-subject study?
- Repeated Measurements
- Each session; observe trends
- At least 2 testing phases:
- Baseline (A phase)
- Intervention
- Target behavior is measured across both phases on multiple occasions
- Baseline phase: state of target behavior over time in the absence of treatment (control conditions)
In a single subject study, what occurs when treatment starts when talking about change?
- When treatment starts, any change from baseline to intervention phase is attributed to the intervention
- Baseline data
- comparison for evaluating potential cause and effect relationship between intervention and target behavior
- Baseline period = A
- Intervention period = B
- A-B design
How do single subject designs differ from traditional experimental designs?
*Multiple assessments in baseline and intervention phases
What are the ethical issues regarding baseline conditions with single subject designs?
*Withholding treatment
What are baseline characteristics?
- 2 baseline data characteristics are important for interpreting clinical outcomes
- Stability (consistency of response over time): stable or variable
- Trend: accelerating or decelerating
What is the length of phases?
- Best to have equal phase length
- Often 1 week per phase
- Daily measurements
- Minimum of 3-4 measurements per phase to detect trends
- Greater number of data points easier it will be to identify trends
- Often measures can be taken more frequently than daily if behavior changes rapidly
- More than a single session
What are Target Behaviors?
- Choose clinically relevant outcomes measures for a particular patient
- Strength
- ROM
- Gait speed
- Balance measures
- Pain
What are the limitations of A-B Design?
- Experiments can control for threats to internal validity
- To do this in the A-B single subject design is more challenging
- Other treatments/events (history effects)
- What other evidence can we include to strengthen design control ?
- to increase confidence that treatment caused the changes in target behavior
What is additional Control?
- *Replication of effects
- Repeat phases
- Withdrawal designs—treatment: no treatment
- Withdrawing and reinstating baseline and treatment conditions
- Withdraw intervention and show that target behavior occurs only in presence of treatment
- -2nd baseline period (A-B-A design)
- -Could also include a 2nd intervention phase (A-B-A-B design—see over)
What is Data analysis- Visual?
- Visual
- Level (last data point of a phase to first data point of next phase)
- Trend (direction of change in a phase)
- -Accelerating or decelerating
- -Slope of a trend (rate of change in the data)
What is a Data Analysis- Celebration line used for?
- Used to estimate trends in the data points
* AKA “split middle line” as it divides the data points in a phase into 2 halves
What is Generalization?
- Single subject research can provide data for clinical decision making
- Not enough to show effect during intervention period on a single patient
- Must also be able to show changes in the target behavior will occur in other individuals
- Generalization: external validity for the single case
- Assume treatment will be effective in others with similar characteristics
What is an observational design?
- No manipulation of variables as in experimental designs
* Exploratory or descriptive
What is exploratory research?
- Systematic investigation of relationships among variables
- e.g. association of leg weakness and falls
- Not used to establish cause-and-effect relationships b/w variables
How can you conduct exploratory research?
*Retrospectively or Prospectively
What is Prospective?
Variables measured in the present and follow subjects in a study
**More reliable than retrospective studies
What is Retrospective?
- Use of data that have already been collected
- medical records, databases
- Researcher can’t control data collection methods
What occurs in Longitudinal Research?
- Follow a cohort over time taking repeated measurements
- Can observe growth and change in individuals over time
- Often involve large cohorts followed over long periods of time
- E.g. Framingham Heart Study (been going 40-50 yrs)
- Threats to internal validity relate to
- repeated testing
- attrition
What occurs in Cross-sectional research?
- Gather data as a “snap shot” in time
- Very efficient
- All subjects tested more or less at same time
What is a correlational study?
- Foundation of exploratory studies is process of correlation (degree of association)
- Covariation in data (extent to which one variable varies with another variable)
- Look at several variables at once to see which are related
- can make predictions (predictive correlational study) based on observed relationships between variables
- -Cholesterol level: age, diet, gender, genetics, etc
- -Regression (stats procedure)
What is the purpose of a correlational study?
*Purpose is to describe the nature of existing relationship among variables
What is a case-control study?
- Retrospective
- not randomized
- Purpose of a case-control study is to determine if the frequency of an exposure (e.g. poor nutrition; smoking) is different in cases and controls
- Choice of controls is critical
- match cases and controls for age, gender, SES, etc.
What is the selection bias in case-control studies?
- Selection bias: choose cases and controls regardless of exposure history
- Beware of misclassification ie cases and controls
What is observation bias in a case control study?
*difference in the way information about disease or exposure is obtained from the groups
What is interviewer bias in a case control study?
*Person collecting data elicits, records or interprets info differently from cases and controls
What is recall bias in a case control study?
*Subjects remember exposure differently
What is a cohort study?
- Follow a group(s) overtime (prospective)
- Group 1: Exercisers
- Group 2: Sedentary (matched)
- Not randomized
What is Causality in Observational studies?
- RCT (experimental): cause-and-effect relationships
- Case control and cohort studies do not involve experiments or manipulation of variables
- Causation (cause and effect i.e. did the exposure cause disease) is established by other methods
What is Causality?
- Establish a time sequence: exposure precedes disease
- Strength of association: relative risk
- Biologic credibility
- Consistency with other studies
- Dose-response relationship-
What is a methodological study?
- Type of exploratory study
* Use correlational methods to examine reliability and validity of measuring instruments
What is a historical study?
- Type of exploratory study
- A historical study reconstructs the past on the basis of archives and other records to suggest relationships of historical interest to a discipline
What populations does descriptive research describe?
- Characteristics
- Behaviors
- Conditions
What does Descriptive research involve?
- May involve prospective or retrospective data collection
- Design may be longitudinal or cross-sectional
- Surveys and secondary analyses of clinical databases often used as data sources for analysis
What are the categories of descriptive research?
- Developmental research
- Normative studies
- Qualitative research
- Descriptive surveys
- Case studies / case reports
What does developmental research involve?
- Involves description of developmental change and sequencing of behaviors in people over time
- motor development in children
- lifespan
What do longitudinal methods of developmental research involve?
*Longitudinal methods involve collecting data over time—focus on natural history of a disease (eg CMT study)
How can you use a cross-sectional study in developmental research?
*Can also use cross-sectional methods and study different age groups at a point in time
What is a normative study?
- Purpose is to describe typical or standard values for characteristics of a population
- Describe norms as a mean and a range of acceptable values
- Norms are used as a basis for prescribing interventions
What is qualitative research?
- Describes how individuals perceive their own experiences within a specific social context
- What it means to live with a spinal cord injury
- Helps us understand the patient’s view of the world
- Important in designing interventions
- Data collected by interviews and observation
- Participant observation (researcher embedded in the group)
- Field observation: nonparticipant
What are descriptive surveys?
- Often used as a source of data to collect information about a specific group
- To describe their characteristics, or risk for disease, or other attributes
What occurs in case studies?
- Important for developing a clinical knowledge base
- In-depth description of a person’s condition or response to treatment
- Case series involves observations in a number of similar cases
- Often involve unusual diagnoses that are challenging
- May highlight avenues for future research
What is the case studies format?
- Comprehensive description of the subjects background, present status, and responses to intervention
- Poses questions for further study
What does the introduction do in a case studies format?
*Describes background literature to the pts problem
what does the pt history do in a case studies format?
*Problems, symptoms, prior treatments, demographic and social info
What are the results in a case studies format?
*pts response and any follow-up data
What is the discussion in a case studies format?
*Interpretation of outcomes and conclusions
What does a case studies format consist of?
- introduciton
- pt history
- treatment plan
- Results
- Discussion
How do you apply literature to patients?
- Systematic Review
- Critically Appraised Topic (CAT)
- -Brief Summary of a search and critical appraisal of literature on a clinical question
- -Standardized format
- -Provides statement of clinical relevance
What is a CAT?
- Initiated by a patient encounter usually due to gap in knowledge
- Search for and appraises best evidence
- Summarizes evidence
- Integrates evidence with clinical expertise
- Suggest how information can be applied to practice
- Usually 1-2 pages
How are CATs and systematic reviews similar?
*They both have the goal of applying their information/findings to people
What is the format of a CAT?
- Title
- Author/date
- Clinical scenario (description of case that prompted the question)
- Clinical question (PICO format)
- Clinical bottom line (summary of how results can be applied)
- Search history/strategy
- Citations
- Summary of the study/ies (design; sample; intervention; outcome measures; data analysis)
- Summary of the evidence (results summarized)
- Critical comments on the study (internal/external and statistical validity of the study)
When do you use CATs?
- Useful at point of care
- Can be created out of case conferences
- “CAT Banks” established by institutions
What are some of the limitations of a CAT?
- Limited shelf life as new evidence becomes available
- Not as rigorous as a systematic review
- -1-2 references and do not represent full scope of the literature on a topic
