Sampling/Validity Flashcards
Population
larger group to which research results are generalized
Examples:
* people with knee osteoarthritis
* elderly with a history of falls
* recreational athletes
* DPT students
Sample
- a subgroup of the population used for estimating characteristics of that population
- More feasible
- More economical
Sample Bias
Choosing a sample that over or under-represents certain attributes may bias the measurement
Examples
* Return to sport after ACL reconstruction:
– Sample = NFL athletes
– Sample = city rec league participants
Goal of Inclusion/Exculsion Criteria
- Goal is to research a sample that accurately represents the population of interest
- Limit the influence of confounding subject characteristics (increases confidence in the study results):
Examples
– comorbities
– concurrent treatment
Inclusion Criteria
description of the traits that qualify someone to be a subject
Exclusion Criteria
description of factors that preclude participation
The more strict inclusion and exclusion results in…
- Less ability for the research to apply to the general public or larger populations BUT…
- Minimizes selection bias and increases the ability to make cause/effect relationships
Types of Sampling Techniques
- Probability Sampling
- Nonprobability Sampling
Probability Sampling
- Randomization involved at some point
- Preferred method
Nonprobability Sampling
- Randomization NOT involved at any point
- Must question the ability to generalize to the population
- Suspect that the sample is biased in some way
- Far more common in clinical research
Simple Random Sampling
- Each member of the population of interest is equally likely to be selected
- Random number generator selects them, requires the entire population to be known
Difficult for studies due to inclusion/exclusion criteria not being able to apply to everyone
Systematic sampling
- Participants chosen from a list (every Kth name)
- Ex: Every 9th name on the list
- Limitation: Requires an entire list of the population.
Stratified Sampling
- Random sampling from subgroups
- Guarantees representation of the entire population, allows for analysis of subgroups seperately
- Ex: Grades of OA in individuals, sorted by severity
Cluster Sampling
- Divide the population into clusters (often by geographical)
- Randomly sample within the cluster
- Measure all nuits within sampled clusters and extrapolate to the entire population
Convenience Sampling
- Use of volunteers very common in PT literature
- Chosen based on availability (clinical site)
- May also be recruited from flyers/signs
- Volunteers tend to have greater motivation
- Especially relevant for experimental studies as volunteers may more strictly adhere to the intervention (Volunteer bias)
- Treatments more likely to show an effect
Consecutive Sampling
- Enrollment of first x number of subjects
- People looking to be involved rather than you looking for them
Quota Sampling
- Non-probability equivalent of stratified sampling
- Subgroups of subjects that vary on some characteristic are recruited/enrolled until appropriate sample size is reached
- Ex: Individuals with knee OA grades I-IV
Not always easy to get all quotas (# of participants) filled for a specific subgroup
Purposive Sampling
- Sample selected for a specific purpose
- Ex: Effect of school size on ACL injury rate
- Commonly used in qualitative studies
Snowball Sampling
- Begins by identifying someone who meets the inclusion criteria for your study
- Ask them to recommend others who they may know who also meet the criteria
Why is having a massive number of participants not good?
A greater number of people in your results may manipulate because we tighten up the numbers and given more room for error. Too much room for error gets rid of the ability to see difference.
Statistical significance does not mean…
it is clinically significant
How do you choose the number of subjects needed for a study?
- In general a larger subject pool is better
- Research design
- Calculations
What does a valid study rely on to prove cause-effect relationships?
- It relys on the ability to rule out effects of extraneous variables
- If these variables are not controlled would be considered confounding variables
Internal Validity
- Amount that a cause-and-effect relationship is free from the effects of confounding variables
- Our confidence that a change in one things caused a change in the outcome
Requirements for Internal Validity
- Temporal Precedence
– Change in outcome must ocur AFTER a change in treatment - Covariation of cause and effect
– The outcome only occurs in the prescence of the intervention - No plausible alternative explanations
- Types of threats to approach
– Single group threat
– Multiple group threat
– Social threats
Single group threat
- Only one group in your study (no control group)
Multiple Group Threat
- Multiple group threats (More than one group in your study but groups not equivalent or not treated equally)
- Differences between groups that affect a cause/effect relationship; influenced by subject selection
Social Threat
- Interaction of subject with the investigator
- Blinding of subjects midigates this threat
Types of single group threats
- History
- Maturation (Changes occur as a result of a passing of time)
- Testing (Repeated test taking may improve test scores
- Instrumentation (Change in instrumentation)
- Mortality/Attrition (Subjects die due to old age)
- Regression to the mean (statistical phenomenon that affects studies based on extreme scores from a single test; outlyers are within their own extreme results in a more normal curve)
Types of Multiple Group Threats
- Selection-history (groups have different experiences between pretest and posttest; Ex: performing the test at different institutions)
- Selection-maturation (groups may naturally change at different rates)
- Selection-testing (pretest affects group differently; people who get pain after test vs no pain effects on performance)
- Selection-instrumentation (Instrumented data is not consistent between groups)
- Selection-regression (stimulus and experience don’t match; high goes low, low goes high)
Types of Social Threats
- Performance Bias (Motivation to suceed is different)
- Contamination (Members of the control group receive intervention)
- Co-intervention (Subjects seek/receive another form of treatment that influences the DV)
- Different Therapists
- Attention Bias (Hawthorne Effect: People’s behavior and performance improves following new or increased attention; Reverse Hawthorne Effect: People’s behavior and performance decline if they perceive they are not recieving equal attention/care.
Threats to External Validity
- People (selection process creates a sample that does not represent the population)
- Place (Dependent variable greatly affected by the setting in which the data were collected)
- Ecological Validity (Study atmosphere resembles real-life
- Time/Technology (Old studies technology not as accurate)
Detection Bias
- Number of variables
– Too few: increase likelihood the treatment effect was missed
– Too many: increase likelihood difference is by chance - Adjustments made to statistics to account for multiple comparisions
– If so: bias favors control group (“conservative”)
– If not: bias favors experimental group (“liberal”) - Recall bias (investigators use self report tools and ask subjects to recall past events)
