Labs - Beth Morling book Flashcards
What is an empiricist?
Basing one’s conclusions on systematic observations
Difference between research producer and consumer?
Research producers – people who actually physically make the data
Research consumers – People who look and the data and apply it
How scientists work: 5 steps
- Act as empiricists
- Test theories based on resulting data
- Follow norms that prioritize objectivity and fairness
- Take an empirical approach to research
- Make their work public
What is theory-data cycle?
Scientists collect data to test, change, or update their theories
What is cupboard theory and contact comfort theory?
Cupboard Theory – Mother-infant attachment in based on a food source being present from the mother to the infant
Contact Comfort Theory – Babies are attached to their mothers because of the comfort of their warm, fuzzy fur
(Harlow’s experiment on monkeys – supported the contact comfort theory as they clung to the cloth mother for 12-18 hours a day)
What is theory, hypothesis and data?
Theory – is a set of statements that describes general principles about how variables relate to one another
Hypothesis – is stated in terms of the study design and the predicted outcome
Data – are a set of observations that may support or challenge the theory
What is preregistered hypotheses, replication and weight of evidence?
Preregistered hypotheses – the researcher states publicly what the study’s outcome is expected to be
Replication – means the study is conducted again to test whether the result is consistent
Weight of the evidence – the collection of studies, including replications, of the same theory
What is falsifiability, universalism and communality?
Falsifiability – a feature of the scientific theory, in which it is possible to collect data that will indicate that the data is wrong
Universalism – scientific claims are evaluated, independent of the researcher’s reputation. The same preestablished criteria apply to all scientists and all research
Communality – scientific knowledge is created by a community and its findings belong to the community
What is disinterestedness and organised skepticism?
Disinterestedness – scientists strive to discover the truth and they are not swayed by conviction, idealism, politics or profit
Organized skepticism – scientists question everything, including their own theories, widely accepted ideas and ‘ancient wisdom’
Difference between applied, basic and translational research?
Applied research – is done with a practical problem in mind and the researchers conduct their work in real-life context
Basic research – is to enhance the general body of knowledge rather than to address a specific, practical problem
Translational research – is the use of lessons from basic research to develop and test applications to treatments
Scientific Journal
- Goes through peer-review (Anonymous)
- Journalism – is a second hand report about the research written by journalists
- Mozart effect – misrepresent research findings and produce exaggerated reports
What is a comparison group? and example
Comparison group: enables us to compare what would happen both with and without the thing we are interested in
- e.g. William Halsted in performing radical mastectomy to cure breast cancer (1700 women were tested and the results showed that the radical procedure didn’t lead to better outcomes)
- Basing conclusions on personal experience is problematic because daily life doesn’t include a comparison group
What are confounds?
Confounds: a potential alternative explanation for a research finding and a threat to internal validity
- Isolate the variables (Bushman in variables affecting anger)
The research versus your intuition –Difference between availability heuristic, present bias, conformation bias and bias blind spot?
Availability heuristic: a type of bias where people incorrectly estimate the frequency of something because they rely on instances that come easily to the mind, rather than using all possible evidence (e.g. being more concerned about cancer as more news stories about it)
Present bias: a type of bias where people incorrectly estimate the relationship between the event and the outcome as they fail to consider the evidence that is harder to notice
Confirmation bias: to consider evidence that only supports a hypothesis and asking questions that will lead to the expected answer
Bias blind spot: people think that they are less likely to engage in biased reasonings compared to others
- You are more likely to trust someone’s statement if that person is an authority figure
Finding and reading the research - What are empirical journal articles and reviewing them
Empirical journal articles: an article that reports for the first time the results of a research study
Review journal articles: summarize and integrate all the published studies that have been done in one research area (could use a quantitative technique called meta-analysis)
What is effect size? and google scholar used for
Effect size: the strength of the relationship between two or more variables
PsycINFO/ Google Scholar: tools for sorting through psychological research
Components of an empirical journal article: 6 features
- Abstract (summary of the article’s hypothesis , methods and major results)
- Introduction
- Method
- Results
- Discussion
- References
Avoiding disinformation: 3 ways
- Motives of disinformation
- Types of disinformation
- Read critically but not cynically
Difference between variables and constant
Variables: Something that varies
Constant: Something that could potentially vary but that has only one level in the study
Types of variables: 4 types
Measured Variable: levels that are simply observed and recorded
Manipulated variable: a variable a researcher controls
Construct variable: name of the concept being studied
Operational variable: how the construct is measured or manipulated in an actual study
Types of claims? 3 types
Frequency claims: describes a particular rate or degree of a single variable
Association claims: argues that one level of a variable is likely to be associated with a particular level of another variable (correlate) can be positive, negative or zero
Casual claim: one variable is responsible for changing the other
Types of validity? 4 types
Construct validity: refers to how well a conceptual variable is operationalized
External validity: how well the results of a study generalise to people besides the original study
Statistical validity: is the extent to which a study’s statistical conclusions are precise, reasonable and replicable
Internal validity: the extent to which a variable, rather than an external cause, is responsible for changes to the other variable
What is covariance and temporal precedence?
Covariance: the extent to which two variables are observed to go together
Temporal precedence: the study’s method ensures that A comes first in times, before B
What is operational definition?
Operational definition: represents a researcher’s specific decision about how to measure or manipulate the conceptual variable
Types of measure: 3 types
Self-report measure: recording people’s answers to questions about themselves in a questionnaire or interview
Observational measure: recording observable behaviours or physical traces of behaviour
Physiological measure: recording biological data
What are categorical and quantitative variables?
Categorical variables: are categories
Quantitative variables: are coded with meaningful numbers
3 types of scale: and examples
- Ordinal scale – when the numbers represent a ranked order (top 10 best-selling books)
- Interval scale – numbers that meet two conditions (Body temperature)
- Ratio scale – numbers have equal intervals (Height)
3 ways to test reliability
- Test-retest – participant will get pretty much the same score each time they are measured with it
- Interrater – consistent scores are obtained no matter who measures the variable
- Internal – participant gives consistent pattern of answers, no matter how the researchers phrase the question
- Correlation coefficient of r
What is face, content and criterion validity?
- Face – It looks like what you want to measure
- Content – The measure contains all the parts that your theory says it should contain
- Criterion – evaluates whether the measure under consideration is associated with a concrete behavioural outcome
What do convergent and discriminant validity both require?
- Convergent and discriminant validity require collecting data to show a measure is correlated more strongly with measures of similar constructs than with measures of dissimilar constructs
What is an experiment? and three types of variables
Experiment: the researcher manipulate at least one variables and measured another
Manipulated variable: a variable that is controlled (independent)
Measured variable: records of behaviour or attitudes (dependent)
Control variable: variable that stays constant
Why experiments support casual claims? 3 reasons
- Covariance (uses control groups to show covariance)
- Temporal precedence (manipulated IV to ensure that it came first in time)
- Internal validity (reduce confounds)
What are design confounds and selection effects?
Design confound: is an experimenter’s mistake in designing the independent variable
Selection effects: when the kinds of pp’s in one level of the IV are systematically different from those in the other
Difference between post-test only and pretest designs
- Posttest- only design – randomly assigned to IV groups and are tested on the DV once
- Pretest design – randomly assigned to at least two groups and are tested on the key DV twice
(random assignment or matched pairs can help establish internal validity in independent-groups designs by minimizing selection effects)
What is repeated measures and concurrent measures design ?
Repeated measures design:
(pp’s are measured on a DV more than once, after exposure to each level of the IV)
Concurrent-measures design:
(pp’s are exposed to all the levels of an IV at roughly the same time)
Pros and cons of within group designs
Pro’s of within-group designs:
- Ensures all the pp’s in the two groups will be equivalent
- Requires fewer pp’s overall
Con’s of within-group designs:
- Have the potential for order effects
- Might not be possible/practical
- Threat of demand characteristics
Interrogating casual claims with the four validities
- Construct validity – evaluating whether the variables were manipulated and measured in ways consistent with the theory behind the experiment (manipulation checks, pilot studies)
- External validity – asking whether the experiment’s results can be generalized to other people/situations/settings (random sampling, replications)
- Statistical validity – precision of the estimate as assessed by the 95% CI (effect size, replications)
- Internal validity – used techniques such as random assignment (looking for confounds, counterbalancing)
Threats to internal validity: 6 threats
(design confounds, selection effects & order effects)
1. Maturation threats (a change in behaviour that emerges more or less spontaneously over time) – prevented by using a comparison group
2. History threats (external factors that systematically effect most members of the treatment group at the same time of the treatment itself, making it unclear whether the change is caused by the treatment received) – prevented by using a comparison group
3. Regression threats (occurs only when a group is measured twice, and only when the group has an extreme score at pretest) – prevented using a comparison group and careful inspection of the pattern of results
4. Attrition threats (when a certain kind of participant drops out) – prevented by removing that participants scores from the pretest
5. Testing threats (a change in the participants as a results of talking a test more than once) – prevented by using a posttest-only design
6. Instrumentation threats (occurs when a measuring instrument changes over time) – prevented by using a posttest-only design
What is observer bias, demand characteristics and placebo effect?
Observer bias: occurs when a researcher’s expectations influence their interpretation of the results
Demand characteristics: when the pp’s guess what the study is supposed to be about and change their behaviour in the expected direction
Placebo effect: occurs when people receive a treatment and really improve because they think they are receiving a valid treatment
Interrogating null effects – (obscuring factors): 2 effects
- Ceiling effects: all the scores are squeezed together at the high end
Floor effects: all the scores cluster at the low end - Measurement error: a human factor that can randomly inflate or deflate a person’s
true score on the DV
Situation noise
Irrelevant individual differences
- Can be counteracted by using multiple measurements, more precise measurements, within-groups design, large samples and very controlled experimental environments
Types of replication: 3 types
- Direct replication – researchers repeat an original study as closely as they can to see whether the effect is the same in the newly collected data
- Conceptual replication – researchers explore the same research question nut use different procedures
- Replication-plus-extension – researchers replicate their original experiment and add variables to test additional questions
Replication projects: 2 types
- One study, many labs: multiple groups if scientist work together to conduct replication studies
- Many labs, many studies: other replication projects coordinate many labs around the world to replicate a variety of studies
Research transparency and credibility
- Psychologists have identified questionable research practices that include:
- Underreporting null results
- P-hacking
- HARKING
(Scientists now promote open data, open materials, and preregistration to strengthen the verifiability and replicability of studies
- HARKING
Must a study have external validity?
- Theory-testing mode – researchers design studies that test a theory, leaving the generalization step for later
- Generalization mode – researchers focus on whether their samples are representative (frequency claims are always in generalization mode)
- Psychological discoveries are not always applicable cross-culturally (only collect data from WEIRD)
