Validity Flashcards
1
Q
Define sampling frame
A
- represents the group of individuals who have a real chance of being selected for the sample
2
Q
What types of subjects are experiments performed on
A
- performed on a representative sample of subjects rather than on the entire population of individuals
3
Q
Describe sampling
A
- investigators need to establish specific inclusion & exclusion criteria for the subjects in studies
- without criteria, there are limits to the generalizability of the study results (this concept is referred to as “external validity”)
4
Q
Define Type I error
A
- occurs when a difference is found in the study sample but there is in fact no difference present in the population at large
5
Q
Define Type II error
A
- occurs when a difference exists in the population at large but the study results reveal no difference in the study sample
6
Q
What is the most common reason for a Type II error
A
- inadequate sample size
- this is also referred to as having low “statistical power” for the study
7
Q
How is an estimation of sample size performed
A
- it is done by performing an a priori power analysis
8
Q
What percentages of Type I and Type II errors are investigators willing to accept
A
- Type I: 5% risk
- Type II: 20% risk
9
Q
Define probability sampling
A
- involves the use of randomization to select individual potential subjects from the sampling frame
10
Q
Define nonprobability sampling
A
- is a method of sampling in which selected subjects are not drawn randomly from the sampling frame
11
Q
Types of probability sampling
A
- random sampling
- systemic random sampling
- stratified random sampling
- cluster random sampling
12
Q
Define random sampling
A
- is a method of sampling in which every potential individual in the sampling frame has an equal chance of being selected for study participation
13
Q
Define systematic random sampling
A
- is method of sampling in which every xth individual out of the entire list of potential subjects is selected for participation
14
Q
Define stratified random sampling
A
- provides a method for dividing for dividing the individual members of the sampling frame into groups, or strata, based on specific subject characteristics
15
Q
Define cluster random sampling
A
- is a process of dividing the sampling frame into groups based on some common characteristics & then randomly selecting specific clusters to participate in the study out of all possible clusters
16
Q
Types of non probability sampling
A
- Convenience sampling
- Purposive sampling
17
Q
Why is random sampling considered superior to non-random
A
- the results of the study are more likely to be representative of the population at large
18
Q
Define convenience sampling
A
- a type of sampling in which potential subjects are selected based on the ease of subject recruitment
- consecutive sampling
- self selection bias
19
Q
Define purposive sampling
A
- a type of non-random sampling
- it entails potential subjects from a predetermined group to be sought out & sampled
20
Q
How can quantitative data be obtained
A
- instrumented devices
- clinician measurement
- clinician observation
- patient self-report
21
Q
What are the 3 types of measurement data
A
1) Categorical (nominal; 1, 2, 3 categories)
2) Ordinal (1 —–> 5)
3) Continuous
22
Q
Describe categorical/nominal data
A
- involves a finite number of classifications for observations
- a numeric value must be assigned to each category
- the order of numbers assigned to each category is inconsequential
23
Q
Describe continuous data
A
- measured on a scale that can continuously be broken down into smaller & smaller increments
24
Q
Describe ordinal data
A
- uses categories
- the order of the numeric classification is of consequence
- Ex. likert scales, in which a numeric value is assigned to each possible response
25
Q
Describe internal validity
A
- refers to the validity of a study’s experimental design
- internally valid if the experiment can conclusively demonstrate that the independent variable has a definite effect on the dependent variable
- if other factors influence the dependent variable & these factors are not controlled for in the experimental design, internal validity may be questioned
26
Q
Describe extraneous factors
A
- internal validity should be thought of as along a continuum rather than as a dichotomous property
- in lab experiments it is easier to control for confounding factors & thus enhance internal validity than it is in clinical trials
- confounding variables are extraneous factors that may result in false relationships
- extraneous factors must be either controlled or quantified
27
Q
Describe validity bias
A
- potential threats to internal validity often involve some sort of bias
- bias may be inherent to either the subjects in the study or the experimenters themselves
- make sure that the subjects in different groups have similar characteristics, by either randomly assigning subjects to treatment groups or utilizing some type of matching procedure
28
Q
Define selection bias
A
- the characteristics that subjects have before they enroll in a study may ultimately influence the results of the study
- ex. age, maturation, sex, medical history, injury or illness severity
29
Q
Define delimitations
A
- decisions that investigators make to improve the internal validity of their studios
30
Q
What 3 entities can be blinded in a study
A
- the subjects may be blinded to whether they are receiving an experimental treatment or a control treatment
- members of the experimental team who are performing outcome measures should be blinded to the group assignment of individual subjects & the values of previous measurements for individual subjects
- clinicians who are treating patients in clinical trials should be blinded to the group assignments of individual subjects
- blinding is important to the internal validity of a subject
31
Q
Describe external validity
A
- relates to the degree to which the results of a study are generalizable to the real world
- the more tightly controlled a study is in terms of subject selection, administration of interventions, and control of confounding factors the less generalizable the study results are to the general population
32
Q
Describe ecological validity
A
- it is an important issue in terms of translating treatments from controlled laboratory studies to typical clinical practice settings
33
Q
What are the 3 types of validity measures
A
- Content validity (outcome comprehensiveness): Face validity
- Criterion validity (outcome comparison): Concurrent and Predictive (high correlation with future criterion)
- Contract validity: Convergent, Discriminant, and Known groups (different outcomes based on groups)
34
Q
Describe face validity
A
- refers to the property of whether a specific measure actually assess what it is designed to measure
- is an important issue in the development of “functional tests” for patients in the rehab sciences
- is determined subjectively & most often by expert opinion
35
Q
Describe content validity
A
- refers to the amount that a particular measure represents all facets of the construct it is supposed to measure
- is similar to face validity but is more scientifically rigorous
36
Q
Describe accuracy
A
- is defined as the closeness of a measured value to the true value of what is being assessed
- should not be confused with precision of measurement
37
Q
Describe concurrent validity
A
- refers to how well on measure is correlated with an existing gold standard measure
- is an important property to be established for new measures aiming to assess the same properties as an existing test
38
Q
Describe construct validity
A
- refers to how well a specific measure or scale captures a defined entity
- stems from psychology but is applicable to other areas of study, such as the health sciences
39
Q
Describe convergent validity
A
- is the measurement property demonstrating whether a given measure is highly correlated with other existing measures of the same construct
40
Q
Describe discriminative validity
A
- is indicative of a given measure’s lack of correlation or divergence from existing measures that it should not be related to
41
Q
Define reliability
A
- refers to the consistency of a specific measurement
42
Q
Describe intratester reliability
A
- is the ability of the same tester to produce consistent, repeated measures of a test (AKA intrarater reliability & test-retest reliability)
43
Q
Describe intertester reliability
A
- is the ability of different testers to produce consistent repeated measures of a test (AKA interrater reliability)
44
Q
Describe ICCs
A
- estimates of reliability for measures of continuous data are often reported as intraclass correlation coefficients (ICCs)
- ICCs are reported on a scale of 0 to 1
45
Q
Describe Pearson’s r
A
- assesses the association between two continuous measures across a sample of subjects
- if as one measure increases in value the second measure also increases incrementally, then Pearson’s r will approach 1
- Pearson’s r indicates that scores on the two measures are highly correlated
- it will not show that the scores of the two measures are systematically diverging from each other
46
Q
Describe precision of measurement
A
- how confident one is in the reproducibility of a measure
- precision is reported as the standard error of measurement (SEM) in the unit of measure
- precision takes into account the ICC of the measure as well as the standard deviation(s) of the data set
47
Q
Describe limits of agreement (LOA)
A
- Bland & Altman recommend that the limits of agreement (LOA) be calculated when two measurement techniques (or two raters) are being compared to each other
- this technique compares the absolute differences between two measurement techniques & specifically looks for systematic error
- if the subject difference is zero, the two techniques are identical
- the LOA represent a 95% confidence interval of the difference between the two measures
48
Q
Describe agreement
A
- estimates of the consistency or reproducibility of categorical data
- intrarater & interrater agreement are defined the same as with reliability measures
- estimates of agreement are reported with the kappa statistic, which also ranges from 0 to 1 with 1 indicating perfect agreement
49
Q
Commonly used nominal statistical test based on number of groups
A
- 1 group, 2 independent groups, and >2 independent groups use X^2 test
- 2 dependent groups uses McNemar test
- > 2 dependent groups uses Cochran Q test
50
Q
Commonly used ordinal statistical test based on number of groups
A
1 group: Kolmogorov-Smimoff 1 sample test
2 independent groups: Mann-Whitney U test
2 dependent groups: Wilcoxon test
>2 independent groups: Kruskal-Wallis ANOVA
>2 dependent groups: Frriedman ANOVA by ranks
51
Q
Commonly used interval/ratio statistical test based on number of groups
A
1 group: t-test of sample mean vs. known population value
2 independent groups: independent samples t-test
2 dependent groups: paired t-test
>2 independent groups: ANOVA
>2 dependent groups: repeated measures ANOVA
