Research Methods and Study Designs Flashcards
What is a dependent variable?
The variable being that is being measured (that is affected by the independent variable)
What is the independent variable?
The variable being manipulated
What is reproducibility?
Refers to the ability of an experiment and its results being reproduced by other experimenters/researchers
What is an operational definition?
a specification of precisely what each variable in a study means
Distinguish between quantitative and qualitative.
quantitative: a numerical value (which allows for statistical analysis)
qualitative: a descriptive value
What does a homogenous control or group refer to?
A group that is the same throughout, and as similar as possible to the experimental group.
What is an extraneous/confounding variable?
variables other than the treatment that could potentially explain an experimental result
What is the main purpose of a double blind experiment?
to help counter the placebo affect
What is sampling bias?
Refers to when not all members of a population are equally likely to be sampled for an experiment. i.e. social research at a university randomly samples undergraduate students for a study (because that’s the most readily available surrounding population), but then applies the results of the study to the general population.
What is selection bias?
- a general category of systemic flaw in a design that can compromise results i.e. sampling bias, attrition bias
- another type of selection bias is purposefully selecting which studies to evaluate in a meta-analysis
What is attrition effects?
Refers to participants dropping out of a study for reasons that are not random; can be caused by participant fatigue. This might introduce an extraneous variable
What is the randomized block technique?
Refers to when researchers evaluate where participants fall along the variables they wish to equalize across experimental and control groups. Then they randomly assign individuals from these groups so that the treatment and control groups are along similar variables of interest
The most important aspects of measurement for an experiment are that the dependent variable is__1__ and that the instruments are __2__
1) quantitative
2) reliable
What does it mean for instruments to have reliability?
means that the instrument produces stable and consistent results
What does replicability refer to?
repeated measurements (i.e. by instruments) lead to similar results
What is construct validity?
the measurements of an experiment measure what they’re supposed to
What is psychometrics?
the study of how to measure psychological variables through testing i.e. how to test mood, memory, attitude, etc.
What is response bias?
the tendency for respondents to have conditions or biases that influence their responses, resulting in them providing untruthful/inaccurate answers
What is between-subjects design?
refers to when comparisons are made between subjects, from one group to another
What is within-subjects design?
refers to when the same group is compared at different time points
What is mixed methods research?
any combination of different research techniques i.e. using both between-subjects and within-subjects design, or using both quantitative and qualitative
Distinguish between Type 1 and Type 2 error. Which one is generally worse to make?
Type 1: to falsely suppose the veracity of a result that does not actually exist (a false positive)
Type 2: incorrectly conclude that there is no effect (aka a false negative)
Type 1 error is generally worse
What is a null hypothesis?
the hypothesis that assumes there is no causal relationship between the variables and any effect that is measured, if there is one, is due to chance.
think of this as the objection hypothesis to the actual scientific/experimental hypothesis
What is the experimental hypothesis in comparison to the null hypothesis?
the proposition that variations in the independent variable cause changes in the dependent variable (which is the opposite of the null hypothesis)
What does it mean when the null hypothesis is rejected?
Rejecting the null hypothesis is proving that there is a measured significant difference in the data, which then means that the actual scientific/experimental hypothesis is supported.
What is a significant difference?
a measured difference between 2 groups that is large enough that it is probably not due to chance.
usually this observed difference is 5% (or 0.05) or less
What is a p-value?
a number between 0 to 1 that represents the probability that a difference observed in an experiment is due to chance.
usually if p <0.05, scientists reject the null hypothesis
What is the “power” of the experiment? What increases the power of the experiment?
Refers to the ability of an experiment to pick up an effect if one is actually present
The larger the sample size, the more power the experiment will have.
Does a lower or higher p-value represent a stronger relationship?
a lower value; the lower the value, the stronger the relationship
Small sample sizes leads to insufficient _____
power
What is external validity?
the extent to which findings can be generalized to the real world
What is internal validity?
the extent to which a researcher can say that the change in the outcome/dependent variable is actually due to the intervention/independent variable (and not due to an extraneous variable or any inherent flaw in the design)
What are demand characteristics?
the tendency of participants to consciously or subconsciously act in ways that match how they are expected to behave i.e. participants interpret what the experiment is about and respond in ways that are consistent with the hypothesis
this is a threat to internal validity
What is impression management?
participants adapt their responses based on social norms or perceived researcher expectations: self-fulfilling prophecy, methodology is not double blind, or Hawthorne Effect
What is the Hawthorne effect?
refers to the tendency of people to work harder and perform better when they are part of an experiment
What is lack of generalizability?
laboratory setups that don’t translate to the real world
How can selection criteria of an experiment threat the validity of the experiment?
too restrictive of inclusion/exclusion criteria for participants will not be a true representative of the population
What are situational effects and how do they threat validity?
situational effects refer to laboratory/experimental conditions that change the outcome i.e. pre-test and post-test, presence of the experimenter, claustrophobia in an MRI machine
what 2 aspects can cause an experiment to lack statistical power
sample size is too small
sample groups have a high variability
_______ design offers the only way to confidently establish a causal relationship between 2 variables
experimental
What is Pearson Correlation?
assigns a number from -1 to +1 to a pair of variables
if the value is negative, the 2 variables are negatively correlated
if the value is positive, the 2 variables are positively correlated
a value of 0 = no correlation
what are correlational studies?
they explore the relationship between 2 quantitative variables
what are ethnographic studies?
a qualitative method in which researchers immerse themselves completely in the lives, culture, or way of life of ppl they are studying. These studies tend to be lengthy and thorough and involve as little interference or intervention by the researchers as possible
What are twin studies?
often test the relationship between nature and nurture
they are the best way to measure heritability (the extent to which a trait is due to genetics and not the environment)
What are longitudinal studies?
involves measuring dependent variables over long time frames
these studies are expensive, hard to execute, and have high attrition rates
What is a cross-sectional study?
involves data collection or looking at data from a population at one specific point in time
what are case studies?
in-depth exploration of one individual or case; explores the potential causes that lead to a phenomenon
what are a phenomenological studies?
uses an introspective method to explore research questions: researchers study themselves through self observation, or researchers record what individuals report about their own personal experience
What are 2 benefits and 3 potential flaws of surveys?
2 benefits: easy to administer and cost effective
3 flaws: unrepresentative sample; respondents may not feel encouraged to provide accurate, honest answers; must be checked for reliability and consistency to be sure they measure the variable they are designed to (i.e. poor survey questions)
what are archival studies?
analyze already collected data from historical records and authentic original documents
What are biographical studies?
exhaustive accounts of an individual’s life experience
What are observational studies?
any study in which individuals are observed and outcomes measured with no attempt to control the outcome
What are threats to internal validity? (5)
- impression management (demanding characteristics, self-fulfilling prophecy, not double blind, or Hawthorn effect)
- confounding/extraneous variables (variables not accounted for in the study or lack of useful control)
- lack of reliability (instruments don’t measure what they’re supposed to or lack consistency),
- sampling bias (i.e. sampling is not random or representative of the population)
- attrition effects (participant fatigue and participants dropping out of the study)
What are threats to external validity? (4)
- experiment/laboratory set-up doesn’t reflect the real world
- selection criteria: too restrictive of inclusion/exclusion criteria for participants and therefore sample is not representative
- situational effects: laboratory conditions changes outcome i.e. pre and post tests, presence of experimenter, claustrophobia in MRI machine
- lack of statistical power: sample size is too small or groups have high variability
What is the ultimate goal of experimental designs and how do researchers accomplish this goal?
to draw causal relationships between 2 variable.
they accomplish this by exerting a lot of experimental control: control vs experimental groups, random sampling and assignment, control of extraneous variables
What is the ultimate goal of non-experimental designs?
they are simply trying to observe and not trying to intervene a study; they thus are not trying to control the outcome
-a flaw usually involves the researchers doing too much i.e. intervening too much to control the outcome
Out of -1, 0, and +1, what is the weakest possible correlation?
0
the closer the value is to 1, the stronger the correlation is
What is the R value?
the correlational value that can range from -1 to +1 (including 0). The value itself will tell you the strength of the correlation. A value of 0.5 is a pretty strong/significant correlation
