Research Methods Flashcards
Scientific Method
- In general an iterative and collaborative process
- Starts with an observation that leads to a theory; based on this theory a hypothesis is created; it is then tested and researched either correlationally or experimentally; the outcomes are reported and either lead to refining of theory or replication and further research
Theory
An integrated set of principles that explain and predict observed events
Hypotheses
- Testable predictions about the relationship between two or more variables
- Written in if-then statements about how variables are (predicted to be) related
Theory refinement
- Mechanisms
- Boundary conditions
- Extensions
What is a good theory?
The best explanation for what we see and find in research
Why is a good theory not categorized as “true”?
- Psychological theories are never proven, as they are inferential (based on subjective matter/target) and evolving (no theory has 100% been completely supported in a fixed way)
- This is applicable to many domains
- Natural selection is a massively powerful explanation of how life evolved on Earth, but it is still a theory which allows that there might be better explanations; like when talking about evolutionary explosions, natural selection does not explain this well initially
- However just because there are clashes between explanations or a theory does not hold 100%, it is not a reason to discredit the theory, which is an invalid tendency
Conceptual Variables
Abstract concept that one may attempt to measure or manipulate (depression, conformity, cohesiveness, aggression, altruism, self-esteem, intelligence, prejudice)
Operational definition
States specifically how the conceptual variable will be manipulated or measured (questionnaire ratings, behaviour, physiological indexes reaction times, etc.)
The Social Psychologist’s Tool Kit
- Self-report
- Reaction Times
- Virtual Environments
- Actual Behaviour
- Biological Measures
Measuring Variables using Self Report
- Based on the simple idea of asking people about their ABCs
- Helps measure subjective and covert conceptual variables
- Presumably best view of psychological processes
- Simple, cost-effective
- However, not always accurate and affected by the way in which questions are asked and responding sets (double barrelled, leading, double negatives, yay/nay saying, fence-sitting, faking good/bad)
Issue with Self-Report (Nisbett and Wilson, 1977)
- Raised question of are people most accurate at determining & answering questions about what & why they feel, act or think so and so
- Reviewed a lot of studies and concluded that people can be and often are:
- Unaware of stimuli that elicit a response
- Unaware of a response
- Unaware of the stimulus linked to response
- Raised another question of whether people can introspect at all, which they can IF stimuli are salient and there are plausible causes of responses, but people like narratives more than what is true
- It was a hugely important paper and was cited 12000+ times
Bias and Desirability in Self-Reports
- Bias creates errors that impact self-reports and cause inaccuracies
- Desirability also impacts self-reports, as people will answer based on how they want to appear to themselves or others even if it is untrue (Modern Racism Scale)
Self-Deception Scale
- Test that assumes the answer is yes to all its items (ex. have you ever felt like you wanted to kill somebody)
- Shows that we can trick ourselves with the report either being wrong or people reporting something more desirable
- Also demonstrated it can be a good thing in small doses, as it promotes performance (swim meet study where swimmers were questioned with this scale and demonstrated negative rltnship (more yes associated with less time in the water)
Measuring Variables Using Behaviour
- We can see it
- Bypass (some) problems of self-report
- Maybe easier to operationalize (like risk-taking with the dollar amount someone bets in a risky game or persistence with length of time working on a tough anagram or even unsolvable)
- Can have real consequences (people care about money, performing well, connecting with other ppl, etc.)
Issues with Behaviour
- Behaviour is a big leap from psychology
- Helping behaviour is altruism or it could also be social sensitivity, low commitment to obligations or other; it is not clear and there can be multiple reasons
- Behaviour can be changed temporarily (same issue as social desirability) especially when observed
Measuring Variables: Reaction Time
Reaction time measures increasingly used in social research (ex. priming tasks, implicit association tasks, go/no-go association tasks, etc.) and provides access to unconscious or automatic processes
Measuring Variables: Biological Measures
- Functional and structural differences in brain and body
- Indicate differences in psychological processes, however it remains inferential as unless if you KNOW the process & how the brain works & the different brain regions that interact with it & its function you cannot establish causality based on a part of the brain lighting up
- Therefore inferential problem is reduced by convergent prior research (skin conductance and arousal for example)
Two Popular Methods of Biological Measures
- Electroencephalography (EEG)
- Magnetic Resonance Imaging (MRI) (or sometimes Functional (f)MRI)
EEG
- Brain activity on scalp
- Look at Event-related potentials (ERPs), by looking at how the brain activity on the scalp shifts when you do or see smth
- Also Frequency-based measures which are how many oscillations are on scalp & how they change throughout time period
fMRI
- fMRI measures blood flow change based on assumption that more blood in an area means more oxygen and therefore more activity
- Can examine changes between conditions
MRI
- Can measure brain anatomy in general, like the synapses and cell bodies in the Cortex or the connections and fibres in the brain in areas of White Matter
- Can see number of connections between two regions
Measuring Variables: Virtual Reality
- Utilizes virtual worlds and simulations
- Participants can engage in tasks that are difficult or impossible to assess in the real world
- However, difficult to set up and analyze, as well as is expensive
Validity and Reliability
- Given the inherent issues, you want to show how ‘good’ your measures are with validity (“Does it measure what you want it to measure”) and reliability (“does it measure the same thing”)
Different Test Validities
Construct, Convergent, Discriminant, and Predictive
Construct Validity
Whether the measures relates to the underlying theoretical constructs
Convergent Validity
Whether the measure relates to other measures it should be associated with
Discriminant Validity
Whether the measure does not relate to other measures it should not be related to
Predictive Validity
Whether the measure relates to another measure of the same construct administered in the future
Can anything be perfectly measured?
What is ‘generosity’ and can it be perfectly measured?
Reliability
- Tendency of a measure to get the same result more than once
- Measurement error, also called error variance, the cumulative effect of extraneous influences
Factors that Reduce Reliability
- Low precision of measurement (Scale of 1-3 vs. 1-100)
- The state of the participant could be dependent on factors outside study
- The state of the experimenter person, behaviour, lack of consistent script
- The environment (temperature, weather noise, etc.)
Reliable not valid
Reliability hitting same spot but not on target
Valid Not Reliable
Measuring what you want but a lot of error
Neither Reliable Nor Valid
Measuring something difference with ton of error
Both Reliable and Valid
All in bullseye
Theory example Ego Depletion: The active self is a limited resource
- Does exerting willpower/control deplete us?
- Theory: People have a common source of controlled resources/willpower from which they draw; Each time we try to control ourselves, we reduce this pool of resources
How can we test this ego depletion theory
- Create a hypothesis: People on a diet have fewer cognitive resources than people not on a diet
- Method: Correlational (observe/measure natural associations to assess the relationship between two or more variables); Experimental (manipulate one variable to see if it affects another variable in a predicted fashion)
Correlational Research
Examine whether the occurrence of A is related to the occurrence of B by measuring both; Results see correlation between variables A and B, but directionality and internal validity are not able to be established
Experimental Research
- Examine whether Variable A causes changes in Variable B by manipulating the former while maintaining all other factors constant (experimental control), then measuring the latter; random assignment is used when assigning participants to treatment or comparison groups
Random assignment
- Eliminates problems with comparing groups
- Randomly placed in experimental (treatment) group or comparison (control) group
- Only difference is manipulation (causality) which allows you to be even more sure that independent variable caused dependent variable
Internal Validity
Whether changes in the independent variable cause changes in the dependent variable without any third variables involved
Construct Validity in Experimental Research
Whether the manipulation of the independent variable is a good representation of the theoretical construct
External Validity
- Whether the results generalize to other labs, participants, settings
- Mundane vs psychological realism (participants are deceived in believing it is real)
Experimental vs Correlational
- Both attempt to assess the relationship between two variables
- The statistics (with the two groups) are interchangeable
- The experimental method manipulates the presumed causal variable, and the correlational method measures it
- ONLY experiments can assess causality, as correlational studies have unknown direction of cause and no manipulation of a SINGLE variable leading to a potential third variable problem
Complications with Experiments
- Uncertainty about what was really manipulated (3rd va problem again)
- Can create unlikely or impossible levels of a variable
-Often requires deception - Not always possible
- Experiments are not always better
Replication Crisis in Social Psychology
- Amy Cuddy’s theory on power posing making people feel and seem more powerful, very influential: unreplicable
- Diederik Stapel, multiple publications, data was too good and unavailable: fake data
- Jens Foerster and super linearity, extremely prominent theory of GLOMO, someone noticed how his studies always looked linear: faked data
- Michael J. Lacour, contact changes minds (could flip people from anti- to pro-gay marriage in minutes and have it last months and even have inter-household spread), statistical irregularities, no error or variability in data: stole data from online CCAP source (ideas were super relevant and later ethically reresearched with similar real results by supervisor)
- Francesca Gino, study on dishonesty, professor at Harvard, Data Canada blog discovered evidence of fraud: faked data on at least 4 or 5 studies and is suing everyone accusing her
Consequences of Replication Crisis
Faked results spurred a deeper look at psychological science as a whole (what can we trust, how do we do research) and lead to a tightening and revision of rules and ethics to diminish questionable research practices (QRPs) and fraud
Open Science Collab
Reran 100s of big effect studies and in general, less than half were actually replicated and most new effect sizes were smaller
Causes of nonreplication
- Small sample sizes
- QRP methods (p-hacking to p < 0.05) by dropping subjects, conditions, dependent variables and file drawers filled with unpublished studies that did not work
Improving our science
- Increase sample sizes
- Direct (vs Indirect) replications by multiple labs
- Preregistration of studies to prevent p-hacking (very important!!)
- Offsite repositories for stimuli (to facilitate replication attempts)
- offsite repositories to store data
The evolution of a theory or field
- Science is a cumulative process
- It depends on direct and conceptual replications
- Science is continually evolving moving toward a more useful and hopefully valid understanding of reality
- People must remain critical
