Validity Flashcards
(51 cards)
1
Q
Define sampling frame
A
- represents the group of individuals who have a real chance of being selected for the sample
2
Q
What types of subjects are experiments performed on
A
- performed on a representative sample of subjects rather than on the entire population of individuals
3
Q
Describe sampling
A
- investigators need to establish specific inclusion & exclusion criteria for the subjects in studies
- without criteria, there are limits to the generalizability of the study results (this concept is referred to as “external validity”)
4
Q
Define Type I error
A
- occurs when a difference is found in the study sample but there is in fact no difference present in the population at large
5
Q
Define Type II error
A
- occurs when a difference exists in the population at large but the study results reveal no difference in the study sample
6
Q
What is the most common reason for a Type II error
A
- inadequate sample size
- this is also referred to as having low “statistical power” for the study
7
Q
How is an estimation of sample size performed
A
- it is done by performing an a priori power analysis
8
Q
What percentages of Type I and Type II errors are investigators willing to accept
A
- Type I: 5% risk
- Type II: 20% risk
9
Q
Define probability sampling
A
- involves the use of randomization to select individual potential subjects from the sampling frame
10
Q
Define nonprobability sampling
A
- is a method of sampling in which selected subjects are not drawn randomly from the sampling frame
11
Q
Types of probability sampling
A
- random sampling
- systemic random sampling
- stratified random sampling
- cluster random sampling
12
Q
Define random sampling
A
- is a method of sampling in which every potential individual in the sampling frame has an equal chance of being selected for study participation
13
Q
Define systematic random sampling
A
- is method of sampling in which every xth individual out of the entire list of potential subjects is selected for participation
14
Q
Define stratified random sampling
A
- provides a method for dividing for dividing the individual members of the sampling frame into groups, or strata, based on specific subject characteristics
15
Q
Define cluster random sampling
A
- is a process of dividing the sampling frame into groups based on some common characteristics & then randomly selecting specific clusters to participate in the study out of all possible clusters
16
Q
Types of non probability sampling
A
- Convenience sampling
- Purposive sampling
17
Q
Why is random sampling considered superior to non-random
A
- the results of the study are more likely to be representative of the population at large
18
Q
Define convenience sampling
A
- a type of sampling in which potential subjects are selected based on the ease of subject recruitment
- consecutive sampling
- self selection bias
19
Q
Define purposive sampling
A
- a type of non-random sampling
- it entails potential subjects from a predetermined group to be sought out & sampled
20
Q
How can quantitative data be obtained
A
- instrumented devices
- clinician measurement
- clinician observation
- patient self-report
21
Q
What are the 3 types of measurement data
A
1) Categorical (nominal; 1, 2, 3 categories)
2) Ordinal (1 —–> 5)
3) Continuous
22
Q
Describe categorical/nominal data
A
- involves a finite number of classifications for observations
- a numeric value must be assigned to each category
- the order of numbers assigned to each category is inconsequential
23
Q
Describe continuous data
A
- measured on a scale that can continuously be broken down into smaller & smaller increments
24
Q
Describe ordinal data
A
- uses categories
- the order of the numeric classification is of consequence
- Ex. likert scales, in which a numeric value is assigned to each possible response
25
Describe internal validity
- refers to the validity of a study's experimental design
- internally valid if the experiment can conclusively demonstrate that the independent variable has a definite effect on the dependent variable
- if other factors influence the dependent variable & these factors are not controlled for in the experimental design, internal validity may be questioned
26
Describe extraneous factors
- internal validity should be thought of as along a continuum rather than as a dichotomous property
- in lab experiments it is easier to control for confounding factors & thus enhance internal validity than it is in clinical trials
- confounding variables are extraneous factors that may result in false relationships
- extraneous factors must be either controlled or quantified
27
Describe validity bias
- potential threats to internal validity often involve some sort of bias
- bias may be inherent to either the subjects in the study or the experimenters themselves
- make sure that the subjects in different groups have similar characteristics, by either randomly assigning subjects to treatment groups or utilizing some type of matching procedure
28
Define selection bias
- the characteristics that subjects have before they enroll in a study may ultimately influence the results of the study
- ex. age, maturation, sex, medical history, injury or illness severity
29
Define delimitations
- decisions that investigators make to improve the internal validity of their studios
30
What 3 entities can be blinded in a study
- the subjects may be blinded to whether they are receiving an experimental treatment or a control treatment
- members of the experimental team who are performing outcome measures should be blinded to the group assignment of individual subjects & the values of previous measurements for individual subjects
- clinicians who are treating patients in clinical trials should be blinded to the group assignments of individual subjects
- blinding is important to the internal validity of a subject
31
Describe external validity
- relates to the degree to which the results of a study are generalizable to the real world
- the more tightly controlled a study is in terms of subject selection, administration of interventions, and control of confounding factors the less generalizable the study results are to the general population
32
Describe ecological validity
- it is an important issue in terms of translating treatments from controlled laboratory studies to typical clinical practice settings
33
What are the 3 types of validity measures
- Content validity (outcome comprehensiveness): Face validity
- Criterion validity (outcome comparison): Concurrent and Predictive (high correlation with future criterion)
- Contract validity: Convergent, Discriminant, and Known groups (different outcomes based on groups)
34
Describe face validity
- refers to the property of whether a specific measure actually assess what it is designed to measure
- is an important issue in the development of "functional tests" for patients in the rehab sciences
- is determined subjectively & most often by expert opinion
35
Describe content validity
- refers to the amount that a particular measure represents all facets of the construct it is supposed to measure
- is similar to face validity but is more scientifically rigorous
36
Describe accuracy
- is defined as the closeness of a measured value to the true value of what is being assessed
- should not be confused with precision of measurement
37
Describe concurrent validity
- refers to how well on measure is correlated with an existing gold standard measure
- is an important property to be established for new measures aiming to assess the same properties as an existing test
38
Describe construct validity
- refers to how well a specific measure or scale captures a defined entity
- stems from psychology but is applicable to other areas of study, such as the health sciences
39
Describe convergent validity
- is the measurement property demonstrating whether a given measure is highly correlated with other existing measures of the same construct
40
Describe discriminative validity
- is indicative of a given measure's lack of correlation or divergence from existing measures that it should not be related to
41
Define reliability
- refers to the consistency of a specific measurement
42
Describe intratester reliability
- is the ability of the same tester to produce consistent, repeated measures of a test (AKA intrarater reliability & test-retest reliability)
43
Describe intertester reliability
- is the ability of different testers to produce consistent repeated measures of a test (AKA interrater reliability)
44
Describe ICCs
- estimates of reliability for measures of continuous data are often reported as intraclass correlation coefficients (ICCs)
- ICCs are reported on a scale of 0 to 1
45
Describe Pearson's r
- assesses the association between two continuous measures across a sample of subjects
- if as one measure increases in value the second measure also increases incrementally, then Pearson's r will approach 1
- Pearson's r indicates that scores on the two measures are highly correlated
- it will not show that the scores of the two measures are systematically diverging from each other
46
Describe precision of measurement
- how confident one is in the reproducibility of a measure
- precision is reported as the standard error of measurement (SEM) in the unit of measure
- precision takes into account the ICC of the measure as well as the standard deviation(s) of the data set
47
Describe limits of agreement (LOA)
- Bland & Altman recommend that the limits of agreement (LOA) be calculated when two measurement techniques (or two raters) are being compared to each other
- this technique compares the absolute differences between two measurement techniques & specifically looks for systematic error
- if the subject difference is zero, the two techniques are identical
- the LOA represent a 95% confidence interval of the difference between the two measures
48
Describe agreement
- estimates of the consistency or reproducibility of categorical data
- intrarater & interrater agreement are defined the same as with reliability measures
- estimates of agreement are reported with the kappa statistic, which also ranges from 0 to 1 with 1 indicating perfect agreement
49
Commonly used nominal statistical test based on number of groups
- 1 group, 2 independent groups, and >2 independent groups use X^2 test
- 2 dependent groups uses McNemar test
- >2 dependent groups uses Cochran Q test
50
Commonly used ordinal statistical test based on number of groups
1 group: Kolmogorov-Smimoff 1 sample test
2 independent groups: Mann-Whitney U test
2 dependent groups: Wilcoxon test
>2 independent groups: Kruskal-Wallis ANOVA
>2 dependent groups: Frriedman ANOVA by ranks
51
Commonly used interval/ratio statistical test based on number of groups
1 group: t-test of sample mean vs. known population value
2 independent groups: independent samples t-test
2 dependent groups: paired t-test
>2 independent groups: ANOVA
>2 dependent groups: repeated measures ANOVA
