Ch. 3: RESEARCH METHODS AND STUDY DESIGN Flashcards
Independent Variable
The variable manipuulated by the research team
Dependent variable
The variable that is measured
Reproducibility in experiments
Good experimental design requires experiments that can be reproduced by other researchers
Operational Definition
A specification of precisely what researchers mean by each variable
Quantitative
Numerical
Qualitative
Descriptive
Experimental group vs Control Group
Experimental group: the group of participants that receives treatment
Control Group: The group that doesnt receive treatment, which acts as a point of reference and comparison
Extraneous (Confounding) variables
Variables other than the treatment that could potentially explain an experimental result
Placebo Effect
The fact that just believing that treatment is being administered can lead to a measurable result
Can lead to erroneous results
Double blind study
Neither the person administering treatment nor the participants truly know if they are assigned to the treatment or control groups
Sampling bias
If it is not equally likely for all members of a population to be sampled
Selection Bias
Purposefully selecting which studies to evaluate in a meta analysis to look for trends in the data
Meta analysis
Big picture analysis of many studies
Attrition
Participants dropping out of the study
Randomized block technique
In this technique, researchers evaluate where participants fall along the variables they wish to equalize across the experimental and control groups
Then they randomly assign individuals from these groups so that the treatment and control groups are similar along the variables of interest
Ex:
List 1: Tall. List 2: Short
What 3 things must a reliable instrument do/have in an experiment?
- They produce stable and consistent results
- Construct Validity: measure what theyre supposed to
- Replicability: repeated measurements lead to similar results
Construct Validity
Measures what its supposed to
Psychometrics
The study of how to measure psychological variables through testing
Response bias
The tendency for respondents to not have perfect insight into their state and provide inaccurate responses
Between-subjects design
The comparisons in the experiment are made between subjects from one group to another
Within subjects design
Comparing the same group of an experiment at different time points
Mixed methods research
Any combination of different research techniques such as within- subjects and between subjects, or qualitative and quantitative
Type 2 Error
False Negative
Researchers accept the null hypothesis, when in fact it is false
Type 1 Error
False Positive
Researchers incorrectly reject the null hypothesis also known as false positive
Which is worse: Type 1 error or Type 2 error
Type 1 error is predominant
A much graver problem in hypothesis testing because it is better to reason from a point of skepticism
This is why scientists start with null hypothesis
Null hypothesis
When scientists assume that there is no causal relationship between the variables and any effect that they measure
Scientists often start with the null hypothesis to prevent type 1 and type 2 errors
Experimental Hypothesis
The idea that changes in the independent variable cause changes in the dependent variable
Significant difference
A measured difference between two groups that is large enough that it is probably not due to chance
This definition is vague on purpose so the researchers must determine when the difference is big enough
P- value
A number from 0 to 1 that gives the probability that a measured difference occurred due to chance
By convention, if and only if p<0.05 scientists reject the null hypothesis
other p values (0.01 or 0.001 are also used as the threshold for significance in some cases
Sample size
The number of participants
Power of experiment
The ability to pick up an effect if one is actually present,
this is related to factors such as large sample size and low variation
External Validity
The ability to apply the conclusion of the research to the real world
Internal Validity
How “well designed” a study is, how valid it is to draw conclusions from the research based on the way it was constructed
Demand Characteristics
The tendency of participants to consciously or subconsciously act in ways that match how they are expected to behave
Predictive validity
The extent to which a psychometric instrument predicts results along a well known test or in a variable of interest
Disclosure
An outline given to participants before the experiment begins that clarifies incentives and expectations while reminding them of their right to terminate the experiment at any time
debriefing
Participants are told after the experiment exactly what was done and why the experiment was conducted
In some cases, if the experiment may have triggered psychological vulnerability, participants may be offered access to treatment or counselling as part of debriefing
What are the main pros and cons of non-experimental research designs?
Pros: Benefit of observing phenomena in a more naturalistic setting, often improving external validity
Cons: Reduced control of the variables of interest, tends ot reduce the internal validity
Correlational Studies
What is it?
Pros and Cons
Explore the relationship between two quantitative variables
Most commonly used type is the Pearson correlation
Advantages:
- Shows numerical relationship between two variables and usually easier to conduct than experiments
- take measurements on population as it exists, no need to directly manipulate variables
Disadvantages:
- causality cannot be inferred
- may not pick up nonlinear relationship
Explain how Pearson correlation studies work
A numerical value between -1 and 1 indicates how two variables correlate
Negative value= inverse relationship
Positive value= direct relationship
Value of 0= no linear relationship between two variables (nonlinear is still plausible)
Values further from zero indicate a stronger relationship (eg. 4 is stronger than 1)
Ethnographic Studies
What is it
Advantages and disadvantages
Qualitative method. Researchers immerse themselves completely in the lives, culture, or way of life of the people theyre studying
Advantages:
- Depth of analysis
Disadvantages:
- researcher is working alone, no one else to critique methodology
- Presence of researcher can affect the findings: makes replication difficult
- Deep immersion in culture can result in feelings of attachment and can threaten objectivity
Twin Studies
Pros and Cons
Studies that incorporate research into different types of twins to gain insight into heritability
Nature vs nurture
Pros:
- insight into nature and nurture
Cons:
- Difficult to find participants who meet criteria
- difficult to analyze the complex variables involved and how they interact
Heritability
The extent to which an observed trait is due to genetics versus the environment
Longitudinal Studies/ Method
Pros and Cons
Measurements at intervals of a dependent variable over long time frames
Pros:
- ability to detail how an effect or factor can develop over time
- allows researchers to have high accuracy when observing for change
Cons:
- Costly
- difficult to execute
- time and resource intensive
- high attrition rates
Cross Sectional study
Data collection or survey of a population or sample at a specific time
related to longitudinal study but slightly different
Case Studies
Pros and Cons
In depth and comprehensive exploration of one individual, phenomenon, or disorder
Pros:
- offer thorough exploration of potential causes that lead to a phenomenon
Cons:
- no isolation of variables or control over the conditions
- results may not be generalizable
- does not offer points of reference or comparison
- difficult to determine how different variables interact
Phenomenological Studies
Pros and Cons
Researchers studying themselves: The use of introspection to explore the nature of phenomena, often related to perception or subjective experience
Attempt to understand own perceptions and understandings rather than make comparisons or draw conclusions
Not all are self observation but sample size is almost always small
Pros:
- detail
- in depth understanding
- insight into behaviours and occurrences that are difficult to measure
Cons:
- lacks objectivity d/t self analysis
- data subjective, affects validity
- Difficult to generalize data
- small sample size reduces external validity
Survey studies
pros and cons
Studies that use questionnaires to explore a research variable
Use to understand how populations feel about topics
Pros:
- easy to administer
- quantitative data that can be compared to large participant pools
- cost effective
- now online, boosts the reach and facility of conducting surveys
Cons:
- Respondents may not feel encouraged to provide accurate honest answers
- can be affected by an unrepresentative sample or poor survey questions
- self reporting creates limitations in objectivity
Archival studies
pros and cons
Studies that explore historical records and search for patterns or insight
pros:
- Provide insight into events from the past that are unique from everyday behaviour
Cons:
- Quality of analysis subject to the quality and integrity of historical records
- difficult to conduct follow ups
- data unlikely to be comprehensive, leaves ambiguity and unanswered questions
Biographical studies
pros and cons
Studies that investigate all relevant details of the life of an individual or small group
Pros:
- Comprehensive knowledge of all details of individuals life
Cons:
- Limitations in objectivity
- difficult to generalize observations
Observational studies
Pros and Cons
A type of research characterized by minimal manipulation in an attempt to investigate phenomena in their naturalistic state
Pros:
- Naturalistic observation of circumstances as they are
Cons:
- difficult to tease out the complex interplay of many variables
A team of researchers measures a negative correlation between income and length of labor time in a group of pregnant women. It is expected that as income increases labor time:
a) decreases
b) increases
c) Stays the same
d) The change in labor time cannot be determined
A: neg correlation means the value measured for one variable increases, the other decreases
Due to scheduling and logistical limitations, researchers conducting a study could only conduct the study on Tuesdays and Thursdays between 7pm and 9pm. they were therefore limited to individuals who were available at this time to participate in the study. This set up would have created issues in:
a) External validity, due to the selection criteria
b) internal validity, due to a potential Hawthorne effect
c) External validity, due to impression management
d) internal validity, due to impression management
A: The recruitment method is problematic because the experiment will not sample individuals who have different schedules and are not available at that time
a sample that is not representative of the population is a threat to external validity. Selection criteria are the processes used to select participants for an experiment.
Researchers want to explore the experience of subjects as they engage in the stroop task, a task in selective attention that measures the ability to distinguish between discordant stimuli. The researchers are not merely interested in response time but want to know qualitative details about the subjects’ internal experiences. Which of the following methodologies should the researchers implement?
a) correlational method
b) observational method
c) survey method
d) phenomenological method
D: The phenomenological method is a technique used to evaluate the experience of some phenomenon and often obtain more introspective details about an event than is possible with other methods. this method is also usually qualitative or descriptive as the question stem suggests
Which of the following is a limitation of the ethnographic method?
a) cultural validity is low, since the observer is not a member of the society of interest
b) External validity is low, because the experimental conditions do not match the real world
c) External validity is low, because only one culture is sampled
d) construct validity is low, since the instruments have not been checked for reliability
C: External validity is an issue in ethnographic studies, mainly because the methodology involves deep exploration of a single culture or subculture so it provides limited info on how the results might apply to other cultures
A team of researchers finds that there is a complex relationship between IQ and sociability. Measures of sociability were found to be high for individuals within one standard deviation of the mean for intelligence, and gradually decreased for individuals with both very high and very low IQ scores. Which of the following correlations would be measured in this instance?
a) Positive
b) Negative
c) No correlation
d) A correlation cannot be determined
C: The results described in the question are curvilinear in nature. They do not represent a linear relationship and would show up as a bell on a graph. Correlational research does not pick up nonlinear trends
In a study of the impact of personality type on the interpretation of a social symbol, which of the following is a possible operational definition of the independent variable?
a) Personality type is the independent variable, so it can be defined as results on a questionnaire designed to measure five-factor model personality traits
b) Personality type is the independent variable, so it can be defined as degree of sympathetic arousal and amygdala activation when seeing a symbol
c) Symbol interpretation is the independent variable, so it can be defined as average ranking, from positive to negative of a series of neutral symbols
d) Symbol interpretation is the independent variable so it can be defined as degree of sympathetic arousal and amygdala activation when seeing a symbol
A: Notice that the first part of the answer choices is divided in half. First determine whether personality type or symbolic interpretation is the independent variable. The study described is designed to measure how personality affects symbolic interpretation. Therefore the independent variable, the variable manipulated by the researcher, is personality type. The five factor model is a common measure of personality. Sympathetic arousal and amygdala activation do not define personality type
Attrition, or subjects dropping out of a study before its completion, is a threat to:
a) Internal validity, because it introduces a potential confounding variable
b) internal validity because the group may no longer be representative
c) External validity, because it introduces a potential confounding variable
d) External validity, because the group may no longer be representative
A: Attrition is primarily a threat to internal validity, because there may be some non-trivial reason that subjects are dropping out. This would present a confounding variable, because if the reason for attrition were related to the hypothesis it could provide an alternative explanation for the results.
It is less likely that enough subjects drop out to threaten the external validity of the study. Also, if participants began to drop out, internal validity would be threatened first, such that it would be difficult to draw a conclusion that would then be applied to the external population.
Which of the following research methodologies would best explore the development of human memory over time?
a) Case study
b) Longitudinal Study
c) Observational study
d) Archival study
