Week 4 Flashcards by Imani B

Validity

the degree to which a test or instrument measures what it purports

How well did you know this?

Not at all

Perfectly

Validity

Logical / Face Validity

do the methods and approach make sense?

How well did you know this?

Not at all

Perfectly

Validity

Content Validity

Does the test fully represent the domain of the concept it is intended to measure?

How well did you know this?

Not at all

Perfectly

Validity

Criterion Validity - Concurrent

correlating an instrument with a criterion administered at the same time

How well did you know this?

Not at all

Perfectly

Validity

Criterion Validity - Predictive

correlating an instrument with a criterion administered in the future

How well did you know this?

Not at all

Perfectly

Criterion Validity - Predictive

Cross- Validation

generate a predictive equation with 1/2 of the sample
confirm the prediction equation works with other 1/2 of sample

How well did you know this?

Not at all

Perfectly

Validity

Construct Validity

degree to which the scores from a test measure the hypothesis construct

How well did you know this?

Not at all

Perfectly

Measurement Reliability

refers to the consistency and stability of a measurement tool or method
indicates how dependably a method measures something over time, across different observers, or in various contexts

How well did you know this?

Not at all

Perfectly

What are common sources of measurement errors?

testing
instrumentation
scoring
participant

How well did you know this?

Not at all

Perfectly

How well did you know this?

Not at all

Perfectly

Reliability Tests

Test-retest Reliability

measures the consistency of results when the same test is administered to the same sample at different points in time

How well did you know this?

Not at all

Perfectly

Reliability Tests

Interrater Reliability

assesses the degree to which different observers or rates agree in their assessments

How well did you know this?

Not at all

Perfectly

Reliability Tests

Parallel Forms

comparing two different versions of a test that are designed to be equivalent

How well did you know this?

Not at all

Perfectly

Reliability Tests

Internal Consistency

examine the consistency of results across items within a test
* a common measure of internal consistency is Cronbach’s alpha, which assesses how well the items on a test measure the same construct

How well did you know this?

Not at all

Perfectly

Common Statistical Approaches

Test-retest Reliability

Pearson correlation coefficient
Intraclass correlation coefficient (ICC)
Coefficient of Variation

How well did you know this?

Not at all

Perfectly

Common Statistical Approaches

Interrater Reliability

Percent agreement
Cohen’s alpha
Intraclass correlation coefficient (ICC)

Common Statistical Approaches

Parallel Forms Reliability

Pearson Correlation coefficient
Split-Half method
Counterbalancing method

Common Statistical Approaches

Internal Consistency

Cronbach’s Alpha
Split-Half reliability

Ranking research design from weakest evidence to strongest evidence.

Animal research (does not involve animals)
Clinical textbooks, experts opinions (not primary research)
Case reports / case studies (no or little design)
Case-control studies (observational)
Cohort studies (observational)
Randomized control trials (experimental)
Systematic review (evidence study)
Meta-analysis (evidence study)

Threats to Internal Validity

History

External or historical event that occurred during the course of the study that may be responsible for the effects instead of the intervention itself

Threats to Internal Validity

Maturation

An internal and natural process that leads participants to change on the dependent measure

Internal Threats to Validity

Testing

Changes in test scores occur because of repeated testing

Threats to Internal Validity

Differential Selection

Any difference between the study groups before the start of the study due to either self-selection or experimenter selection

Threats to Internal Validity

Instrumentation

Changes in instrument calibration or human observers

# Threats to Internal Validity Regression to the Mean

The phenomenon that if a variable is extreme on its first measurement, it will tend to be closer to the average on its second measurement

# Threats to Internal Validity Selection Interactions / Expectancy

Both researchers or study subjects may tend to expect one group may perform better (or worse) the other

# Threats to Internal Validity Experimental Mortality (Attrition)

People can drop out of the study for any reason; if reason is related to treatment status it may cause bias

What are threats to External Validity?

* selection bias * pretesting * experimental setting * multiple treatment

# Threats to External Validity Selection Bias

* Who is included/excluded * Race/ethnicity, age, sex, lifestyles, SES * Disease characteristics

# Threats to External Validity Pretesting

If you pretest, then when participants take the "true test" their responses may differ compared to if they never took the a pretest

# Threats to External Validity Experimental Setting

* Participants performance improves just b/c they know they are being watched * Hawthorne Effect

# Threats to External Validity Multiple Treatments

Receiving on treatment may influence how you respond to other treatments

Random Control Trial (RCT)

* A study design where participants are randomly assigned to either the intervention group or the control group * randomization minimizes bias, allowing for causal inferences

Quasi-Experimental

* A study design that looks at the effect of an intervention without random assignment * while it can still compare groups, the lack of randomization makes it more prone to bias

Natural Experiment

* A study that takes advantage of "naturally occurring circumstances in which subsets of the population have different levels of exposure to a supposed causal factor, in a situation resembling an actual experiment" * Provides an alternative when it is impossible or unethical to assign treatment