Lecture 3 Flashcards
What type of research design is this:
“participants are randomly assigned to at least 2 comparison groups”?
Experimental
What type of research design is this:
“no random assignment & typically involves the use of cohort groups”?
Quasi-experimental
What is key to true experimental design?
Randomization
Repeated measures design is when ___________.
What is the flaw with repeated measures?
Participants serve as their own controls
Flaw: Participants will learn and adapt over time
Which research design is this?
Compares 1 group over 2+ time periods
One-way repeated measures
- intervention can be given at any point or over a few time periods
Which research design is this?
One-way repeated measures
Which research design is this?
Posttest-only randomized group
Note: can be more than 2 groups
Which research design is this?
Posttest-only non-randomized group
Which research design is this?
2+ IVs, with or without randomization into groups, AND the patients never cross into other groups
Factorial - fully independent
Which research design is this?
Factorial - fully independent
Which research design is this?
2+ IVs, the subjects participate in every cell
Factorial - fully repeated
Which research design is this?
Factorial - fully repeated
Which research design is this?
2+ IVs, with or without randomization, groups of subjects move through the repeated conditions
Factorial - mixed
Which research design is this?
Factorial - mixed
Which research design is this?
Crossover with or without randomization
What is measurement theory also known as?
Psychometrics
What does measurement theory provide?
the foundation for evaluating tests, their uses and interpretations
e.g. reliability, validity
What is a methodological study?
Any study that investigates the reliability and/or validity of clinical and research measures
What is MDC?
Minimally detectable change
* amount of change in an instrument that is beyond error
What is MCID?
Minimal Clinically Important Difference
* amount of change in an instrument that results in a clinically relevant change in the patient
What is this an example of?
TUG: change in performance of >2 seconds is true change, not measurement error
MDC
What is this an example of?
TUG: a decrease in time of >5 seconds is important change to the pt diagnosed with MS
MCID
What is reliability?
Consistency or degree of association between 2 variables
What is validity?
Accuracy or degree of correspondence between the concept being measured and variable used to represent the concept
reliable or valid?
reliable
reliable or valid?
valid, not reliable
an instrument can be ______ without being _____, but can’t be ______ without being ________
an instrument can be reliable without being valid, but can’t be Valid without being reliable
Different raters getting the same score is what kind of relaibility?
Inter-tester reliability
The same rater getting the same score is what kind of reliability?
Intra-tester reliability
Can successive measurements by an instrument be consistent? - is what kind of reliability?
Test-retest reliability
Can successive measurements by a patient be consistent? - is what kind of reliability?
Test-retest reliability
Parallel forms reliability
AKA equivalent forms reliability
* questions are divided into 2 equivalent forms
e.g. SAT: create assessments that have similarly difficult items using this method
Split-half reliability
- questions are split into 2 sets, and both sets are given to one group of individuals
e.g. Depression questionnaire with 100 items, someone with minimal depression would score low on both sets of questions
What type of validity is this?
Does an instrument measure what it is supposed to measure based on simple observation?
Face validity
What is content validity?
systematic examination of an instrument to determine whether it covers the entire domain to be measured
What type of validity is this?
e.g.
Does an IQ questionnaire have items covering all areas of intelligence discussed in the scientific literature?
Content validity
What is criterion-based validity?
The degree to which the outcomes of one test correlate with outcomes on a gold standard test
T test vs ANOVA
T test compares 2 sets of data
ANOVA compares many
What is the problem with running repeated T tests for a data set with more than 2 groups?
Inflated alpha level = more chance of type 1 error
What is concurrent validity?
The degree to which the outcomes of 1 test correlate with outcomes on another test, when both are given at relatively the same time
Which has greater/stronger validity?
Face validity or concurrent validity
Concurrent validity
What is predictive validity?
Can an instrument be used to predict some future performance/outcome
What is construct validity?
The degree to which a theoretical construct is measured by an instrument
What type of validity is this an example of: Berg Balance Scale: score of <45 indicates individuals may be at greater risk of falling?
Predictive Validity
What is responsiveness to change?
the extent to which significant changes are reflected in the observed values
What 2 things limit responsiveness to change?
Ceiling & Floor Effect
If a person has a disease or condition, _________ tells you how often the test will be positive
Sensitivity
If a person does not have a disease or condition, ________ tells you how often the test will be negative
Specificty
If Monica tests POSITIVE for the COVID test, she is very likely sick. But if Ben tests negative, he is HIGHLY UNLIKELY to be sick.
This test has high ______
Sensitivity
If Judy tests NEGATIVE for an allergy test, she is most likely NOT ALLERGIC. But if she tests POSITIVE, she is HIGHLY LIKELY to have an allergy.
This test has high _____
Specificity
What is the sensitivity equation?
(True Positive) / (True Positive + False Negative)
What is the specificity equation?
(True Negative) / (True Negative + False Positive)
What is prevalence?
In this population, how common is this disease/injury?
What is the Positive Predictive Value (+PV) equation?
(True Positive) / (True Positive + False Positive)
What is the Negative Predictive Value (-PV) equation?
(True Negative) / (True Negative + False Negative)
What does +PV mean?
What is the probability that a person with + clinical test truly has the condition?
What does -PV mean?
What is the probability that a person with a - clinical test Truly doesn’t have the disease?
When do we need to adjust the predictive values for prevalence?
When your sample doesn’t represent the true prevalence of the condition in your population
+PV increases as the prevalence of a disease in a population _______
Increases
-PV decreases as the prevalence of a disease in a population ________
Decreases
What is the +LR equation?
Sensitivity / (100 - specificity)
What is the -LR equation?
(100 - sensitivity) / Specificity
+ Likelihood Ratio
Best ____
OK ____
Poor ___
Useless ___
Best = 10
OK = 5
Poor = 2
Useless = 1
(-) Likelihood Ratio
Useless ___
Poor ___
OK ___
Best ___
Useless = 1
Poor = 0.5
OK = 0.2
Best = 0.1
For a ROC curve, what is on the x- axis? y-axis?
X-axis is (1-specificity)
Y-axis is Sensitivity
What does an increased area under the line of the ROC curve mean?
High sensitivity and specificity
