Research Design (via G-money's Study Guide) Flashcards
What are the 4 variables that take on MANY values?
- Independent Variable
- Dependent Variable
- Confounding Variable
- Quasi-Independent Variable
what variable is manipulated by the researcher?
Independent Variable (its the presumed agent of change….. you should know this)
what variable is MEASURED by the researcher?
Dependent variable
DV determines if IV has an effect… duh
what variable reduces internal validity?
confounding variable
this variable is an extraneous variable. It varies systematically with the IV
An experiment that uses existing groups (rather than random assignment) to determine condition, is an example of what kind of variable?
quasi-independent variable
What are the 4 measurement scales?
- nominal variable
- ordinal variable
- interval variable
- ratio variable
What is the definition of a nominal variable?
a label or category (e.g. political party)
what is an ordinal variable?
data that is ranked or possess order (e.g. Class rank)
What is an interval variable?
ranked, meaningful differences between values (e.g. Fahrenheit temp scale)
what is a ratio variable?
ranked, meaningful differences between values AND the values of 0 signifies absence of what is measured (e.g. Kelvin temp scale)
Effect size measures determine WHAT?
practical significance rather than statistical significance:
“is the effect large enough to matter?” vs “does effect exist?”
what are 2 things we gotta know about Effect Size?
- used in meta-analysis to combine findings from multiple research studies (due to independence from sample sizes)
- “Specific Effect Size measure” varies by CONTEXT
Proportion of variation in one variable accounted for by the linear relationship with another is called………. ?
Correlation: r squared
difference between two group means in terms of standard deviation (control group or pooled) is called…….. ??
T-test: Cohen’s D
proportion of variation in the DV accounted for by the IV is an……..?
ANOVA: eta-squared, omega-squared
What are 2 things ya gotta know about Mixed-Method Design?
- Its both Qualitative and Quantitative data
2. Addresses limitations of one type with strengths of the other
what kind of design can have a narrowed focus of quantitative data complemented by “big picture” of qualitative data?
Mixed-Method Design
what kind of design allows for a larger, more representative sample, AND adjusting for extraneous variables?
Mixed-Methods Design
limited generalizability of qualitative data covered by external validity of quantitative data
What is a Solomon Four-Group Design?
Its a design that controls for practice effects BY randomly assigning participants to 4 groups
What are 5 things we gotta know about a Solomon-four group design?
- Experimental groups take a post test and receive intervention (and take a pretest if applicable)
- control groups do not receive intervention but take a post test (and pretest if applicable)
- may determine whether practice effects exist by searching for differences across testing levels
- rich results from replicating experimental and control conditions
- requires extra time and money for more groups
What does a correlational (or observational) study examine?
the relationships between unmanipulated variables
What kind of study measures association and does NOT establish cause and effect?
(ex: x may cause y, y may cause x, OR a third variable Z drives both)
correlational (or observational) study
What happens in a double-blind design?
Participants and experimenters are blind (i.e. naive) to experimental condition
What are 4 things that happen in a double-blind design?
- experimenters may unconsciously influence participant behavior to fit research hypotheses
- subtle cues such as tone of voice or posture may bias research results to fit expectations
- an experimenter ignorant of experimental condition should be unable to influence in a manner of fitting expectations
- Variations
In what kind of study is the Participant is naive to condition and the experimenter is not?
single-blind study
in what kind of study is the experimenter, participant AND others involved in research naive to condition?
triple- blind study
the extent to which research findings may be extended to the people, places and situations is called what?
External Validity
Developmental Research assesses changes over a period of time and consist of WHAT 3 designs?
- Longitudinal
- Cross-sectional
- Cross Sequential
What are the 4 main threats to a Longitudinal study?
- lengthy time requirement
- subject morality due to numbers of factors (illness, relocation etc)
- lack of randomization
- history primary threat to external validity
What kind of design has groups at each level measured at the same time?
Cross-sectional
What is a Cross Sequential Design?
combination of longitudinal and cross-sectional designs
what is 1 pro for a longitudinal study?
can provide valuable qualitative and quantitive data
What are 3 things you gotta know about a Cross-sectional design?
- requires much less time than a longitudinal study
- assumes difference reflect natural development
- differences may be due to cohort effect
What are 4 things you should know about a Cross Sequential design?
- different groups assessed repeatedly over time
- reduces time required to preform
- minimizes assumptions/cohort effects
- preform lit review for earlier findings
what is the definition of Statistical Significance?
Results of an analysis reflecting more than one chance variation, concluding an effect most likely exists
What is P-value
conditional probability of obtaining results, if null hypothesis of no effect is true
What do we call the maximum likelihood of making a type I error that will be accepted?
Alpha
what happens if P is greater than alpha?
results are too likely to be due to change and NO effect is concluded
What happens if P is LESS than alpha?
results relatively unlikely to be attributable to chance alone and the existence of an effect IS concluded
P-value is compared to conventionally determined Alpha to determine WHAT?
statistical significance
What is a type I error?
concludes an effect visits when actually obtained results were due to chance alone
What determines a probability of committing a Type I error?
Alpha
What determines a probability of committing a Type II error?
Beta
What is a Type II error?
concludes no effect exists when actually there IS an effect
What do we call “the probability of concluding an effect exists given that it does”?
Power
What are 6 of the ways Power can be increased?
- Increasing effect magnitude
- decreasing error variation
- increasing sample size
- increasing Alpha
- power analysis
- running a one-tailed or directional test
What is random selection?
drawing a sample form a population in such a way that each member has an equal probability of being selected
why is random selection important?
Supports External Validity
What is random assignment?
assigning participants to experimental condition in such a way that there is an equal chance of appearing in any given condition
why is random assignment important?
-supports internal validity
What happens if random assignment is not possible because of pre-exisiting groups?
Quasi-experiment is in order
what affords the best opportunity of obtaining representative samples?
random sampling
What is a sample size?
number of observations (e.g. People) in a sample, denoted by N
What is a “Sample” for the purpose of research?
a subset of a population meant to provide small snapshot that represents the whole body
what are 3 reasons that a Larger Sample Size is preferred?
- Ensure sample is adequately representing the population
- Reduce Sampling error
- Increase Statistical Power (with less error)
What is the definition of an ABAB design?
A single-subject design in which a baseline measure of the dependent variable is obtained (A) before treatment is introduced (B), removed (A) and reintroduced (B)
In an ABAB design, if treatment has an effect, the DV will do 3 things. What are they?
- deviate from baseline when it is introduced
- return to baseline when removed
- deviate again from baseline when reintroduced
(In an ABAB design) what happens if the DV does NOT return to baseline after removal of treatment?
the initial devotion may be attributed to a confounding variable
Why is treatment reintroduced after second baseline in a ABAB design?
- further establish treatment effect
- restore benefit of treatment to subject
What kind of design (that is common in basic research) does NOT reintroduce treatment after second baseline?
ABA design
What kind of design may be altered to include different treatment?
ABAB Design
What is a counterbalanced design?
A design where multiple treatments are administered to each participant in different sequences to prevent order effects
what are 2 disadvantages of a counterbalanced design?
- may require more participants
2. may require more sophisticated analysis
what is an advantage of a counterbalanced design?
may analys between-subject variable of treatment sequence
what is used more often:
A double blind design or a counterbalanced design?
double-blind
In what kind of design does larger # of treatment = larger # of sequences?
Counterbalanced Design
in a counterbalanced design, treatments might not appear in a given position of equal number of times, if the design has what?
a small sample
What does an analysis of 3 different treatments look like in a counterbalanced design? (A, B, C)
ABC ACB BAC BCA CAB CBA
what is a latin square design (via counterbalanced design) ?
where you select a smaller # of sequences in which a given treatment appears only once in each position for 3 treatments
(e.g. ABC, CAB, BCA. A,B,C each appear only once in 1st/2nd/3rd positions)
what is internal validity?
the extent to which research study rules out all alternative explanations and establishes causality
what are the 3 supports of internal validity
- random assignment
- matching
- blocking
What are the 9 threats to internal validity?
- history
- maturation
- testing
- instrumentation
- regression to the mean
- selection
- interactions with selection
- morality
- ambiguity about the direction of causation
If you arnt seeking to establish that A causes B, then what is NOT a concern?
Internal Validity
Internal Validity is of PARAMOUNT importance if you want to establish what?
That A CAUSES B!
What is Cluster Sampling?
a sampling technique that involves naturally occurring groups (clusters)
(i.e. only elements of randomly selected clusters are studied)
what is stratified sampling?
a sample drawn from each stratum
*Main objective is to improve precision
in which sampling is the main objective to improve sampling efficiency and reduce cost?
cluster sampling
In cluster sampling, what should the “clusters” be?
-internally heterogenous YET relatively homogenous among themselves
In cluster sampling, Do you study information from ALL elements within clusters included in the sample?
Yes.
What happens in a matched-subjects design?
each participant in one sample is matched with a participant in another sample, with respect to a specific variable (e.g. SES)
What type of design renders conditions nearly equivalent on matched variable?
matched-subjects design
What does a Matched-subjects design do to increase the Power of the study?
reduce error variance
What does a Matched-Subjects design do that might DECREASE External Validity?
not generalize to others NOT demonstrating the matching variable
