Research methods Flashcards
Introduction to RM, descriptive stats, presenting data, research approaches, organisational designs, probabilities (310 cards)
1
Q
methodology
A
the study of methods
2
Q
method
A
how to reach a certain goal
3
Q
epistemology
A
what is knowledge
4
Q
ontology
A
what is reality
5
Q
philosophy of science
A
what is science?
6
Q
why do i need research methods?
A
limit bias of perception, imperfect memory (cognitive),
prior experience and learning,
create a scientific method
7
Q
challenges in psychological research
A
- unobservable object of investigation
- subjectivity of object of investigation
- social construction (phenomena occurs in environment)
- ethics
8
Q
types of research
A
basic research - to increase the stock of knowledge.
applied research - to increase the use of knowledge to devise new applications.
9
Q
stats in psychology
A
clinical, diagnostics, development, neuroscience and cognition
allow for discovery, modelling and scientific proof.
10
Q
the difference between a belief and knowledge
A
belief- subjectively true
knowledge- objectively true
the research method is a way to check for proofs and obtain knowledge
11
Q
scientific research
A
building knowledge using specific research methods to check for proofs
should be
- transparent
- share expertise
12
Q
hypotheses vs conclusions
A
a hypothesis is an unproven provisional statement
a conclusion is a proven proposition
13
Q
scientific proof
A
logical proof- does it make sense (rationalism)
empirical proof- is there evidence (empiricism)
the process - logical arguement - empirical test - logical argument
14
Q
logical argument (causal influence)
A
3 types of causal influence:
- necessary cause- the cause is to produces the effects
- sufficient cause- the cause alone produces the effect
- contributory cause - this contributes to the cause producing the likelihood or strength.
15
Q
criteria of scientific propositions
A
- logical consistency
- testability
- scope
- fruitfulness
- novelty
- simplicity
- conservatism
16
Q
logical consistency
A
3 types of logical inference:
- deduction - infer info about a single case
- induction - infer a general statement from statements about single cases
- abduction - infer cases the most likely best explanation
17
Q
testability
A
how well it can be tested
18
Q
scope
A
general validity
19
Q
fruitfulness
A
implications beyond the research question
20
Q
novelty
A
information which is new and creating propositions
21
Q
simplicity
A
parsimony - minimise assumptions by creating the most simplistic and scientific explanation
22
Q
conservatism
A
minimise new assumptions that contradict existing knowledge. propositions that integrate with existing knowledge and more likely because it is unlikely to be wrong.
23
Q
theory before the empirical test
A
theory - research question - assumption - hypothesis - prediction
24
Q
theory after empirical test
A
data analysis - conclusion - implications.
25
level of measurement of data
nominal, ordinal, interval or ratio
26
nominal data
categorical data of groups, have to be mutually exclusive (not part of two groups), can’t order the data
27
ordinal data
orders people, objects or events along some continuum (various rankings), no information is given about the differences between the points on then scale
28
interval data
equal intervals between objects represents equal differences, interval scales do not talk about ratios (zero is arbitrary)
eg temperatures
29
ratio data
ratio data has a true zero point eg the absence of something being measured eg weight
30
descriptive statistics
goal to characterise a numerical set of data efficiently and representatively, condense and render meaningful information, minimise error involved in condensing information
31
inferential statistics
the goal is to infer the characteristics of the population from a sample so generalise the results
32
measures of central tendency
mean, median and mode
33
mean
the average score, add up all the scores and divide by the number of scores. mean can be influenced by the anomalies. a histogram can show whether the mean is a good measure
34
median
middle score when all do the scores are rearranged in order from smallest to largest, not affected by extreme scores
(N+1)/2
35
mode
the value with the most frequent score, if two adjacent scores find the middle eg 4 and 5 = 4.5, if two non adjacent scores report the, both eg 4 and 7 (bimodal distribution)
36
measures of variability
the degree to which the values vary: range, interquartile range, variance and standard deviation
37
range
the difference between the maximum and minimum scores, measure of distance, easily distorted by outliers
38
interquartile range
percentiles - cut off that divides the data into percentage chunks
percentiles can try to avoid anomalies. 50th percentile splits the data into percentage hard so 50% scores above and below.
0.5*(N+1) = 50th percentile
0.75*(N+1) = 75th percentile
39
variance
how much the mean scores vary given in
terms of the distance from the mean
the average of each score’s squared deviation from, the mean score
40
sample vs population
population- use the population formula (divide by N)
sample- use the sample formula (divide by N-1)
41
experimental control
casual effect - is what’s wanted to be measured
noise - unwanted
confounders - unwanted
42
noise
random variation which is uncertain
43
signal
systematic, regular and informative
44
bias
unwanted or unexpected signal (e.t. systematic variation, systematic tendency) producing spurious results
consequence- measurements without (real) experimental effects differ from what you assume and expect. discrepancies between results and facts
45
types of bias
independent bias and dependent bias
46
independent bias
systematic errors,
systematic tendencies (signals) in measurements that are incorrect but limited to one variable
bias does not vary with the independent variable hence can’t be mistaken for causal effect (results)
47
dependent bias
cofounder
a confounder produces bias in the dependent variable (DV) that varies with the experimental conditions (IV)
this bias is correlated with both independent and dependent variables - interfere with causal effects (results)
produces an unwanted, spurious effect on measurements (DV)
48
systematic errors
unwanted signal bias that does not interfere
49
criteria of empirical evidence
noise - reduce reliability, reduce accuracy
systematic errors - reduce accuracy
confounders - reduce internal validity and reduce external validity
50
sources of noise and bias
- method (stimulus material and procedure)
- participant (variation of physical condition e.g. fatigue, perception, emotion and cognitive processing)
- data recording (variation of experimenter or measurement instruments)
51
minimise systematic errors
easy to fix or compensate through CALIBRATION - comparison with and adjustment measure
52
why control noise?
noise decreases the accuracy and reliability of measurements:
- results of single measurements are inaccurate due to random variation
- more random variation the less likely it is to get the same results across measurements
53
aim of controlling noise:
maximise systematic variation (signal) due to causal effects relative to random variartion (noise) - single -to -noise -ratio
want to maximise the ratio
54
how to control noise?
- minimise (aka maximise precision)
- neutralise (e.g. maximise sample size)
55
minimise
- instruments with high precision
- tasks with high precision
56
neutralise
take an average of the frequencies - approach the frequencies based on probability
57
neutralise by sample size
if noise is uniformly random, increasing the sample sizes makes the average coverage to the expected value
expected value of noise is the same for all conditions
central tendency of larger datasets involve less noise
58
how to neutralise by sample size?
increase number of participants
increase number of measurements per participants
59
noise and statistical reliability
signal-to-noise-ratio
- the amount of desired signal (e.g. experimental effects) relative to the noise
- the higher the SNR the higher the sensitivity and the specificity of your measurement
60
agency
human beings (agents) act upon and shape their environment
61
interpretation
people perceive, think and make choices to adapt their actions and behaviour
62
interaction
human beings act on other acting human beings - they mutually influence their behaviour
63
double-interpretation
one person interprets how another person interprets and acts accordingly, which may in turn influence the interpretation and action of the other person
64
social construction
interpretation and meaning exist because of the coordination with others rather than separately within each individual, based on mutual expectations
65
double-hermeneutic
concept from sociology, Giddens, 1970
participants interprets the researchers behaviour and setting of the study and adapt their behaviour
66
agency in psychology
- investigators influence participants in studies
- participants influence the interpretation by the investigator
- not limited to direct communication
67
agency-based biases in research
1. participant effects
2. investigator effects
68
participant effects example
Hawthorne studies
- aim to increase performance of factory workers
- manipulation - changing lighting, cleaning workstations, clearing floors and relocating workstations
Hawthorne effect
- productivity gain as a result of increasing worker motivations because of the interest being shown in them
69
participant effects: general idea
participant reactivity
- the act of doing the research changes the behaviour of participants
- due to awareness of the, being observed by the researcher or by other participants
- response bias (unconsciously it could happen)
response bias
- systematic tendencies of participants to respond inaccurately or falsely, producing either a systematic error, or a confound (if mixed with investigated effects)
- participant reactivity - only one of different types of response biases due to the location of response keys
70
types of participant effects
- participant expectancy
- demand characteristics
- social desirability
- stereotype threat
71
participant-expectancy effects
participant expects a result and therefore unconsciously affects the outcome or reports the expected outcome
placebo effect
positive expectations about a treatment improve patient-reported outcomes
72
placebo effect
placebo is a substance or treatment, which is designed to have no therapeutic value
73
Nocebo effect
negative expectations about a treatment cause negative effects
74
demand characteristics
participants form an interpretation of the study’s purpose and subconsciously change their behaviour to fit that interpretation
please-you effect
- ppts try to fulfil the expectations of the researcher to please them
screw-you effects
- ppts are defiant and try to produce unexpected results that screw up the study
75
social desirability bias
tendency to answer questions in a manner that is expected to be viewed favourably by others and that produces or maintains a publicly acceptable image
examples:
- bradley effect: consider voting for a black candidate heavyset it is socially desirable but in the end they don’t vote for them
- evaluation apprehension: when being observed ppts feel evaluated and try to convey a positive image
- watching-eye effect: when being observed, ppts try to behave better than without observation
76
stereotype threat
responses are biased to conform to stereotypes about the ppts social group
negative effects:
negative stereotypes about your own group produce self-doubts and negative expectations towards yourself
positive effects:
stereotype boost: perceive yourself better because of positive stereotypes about your own group
sterotype lift:
perceive yourself better because of negative stereotypes about an outgroup
77
control of participant effects
1. minimise
2. assess
3. account for
78
minimise participant effects
1. blind procedures
- information is withheld until after the experiment
- unobtrusive manipulations and measures: conceal IV and DV so no clue on hypothesis
unobtrusive methods:
-unobtrusive observational data recording
- unobtrusive observation
ethical issues:
- no consent
- risks to ppt and researcher
deception: deceived ppt about one or more aspects of the research to conceal the hypothesis
79
assess participant effects
1. post-experiment questioning
- did they know what was expected
- questionnaires e.g. Perceived Awareness of the Research Hypothesis (PARH) scale
80
account for participant effects
1. control conditions
- everything is the same as in the experimental condition except the experimental manipulation
- baseline results: reference measurement for comparison with experimental conditions
control group
- everything same as in experimental condition except IV
randomised control trials (RCTs)
- typically ppts are randomly allocate to the groups in a randomised control trial - if not quasi
placebo control groups:
- create positive expectations with placebo to establish comparability with treatment groups
81
investigator effects
influence of researcher on results
primary observer effect- effects on participant responses
secondary observer effect- effects on data acquisition and interpretation
82
primary observer effect
researchers expectations about the findings of their research are conveyed to ppt and their responses
examples: experimenter-expectancy and self-fulfilling prophecies
83
clever hans effect
shows the primary observer effect
the horse could “count” but it was just the horse responding to the involuntary cues in the body language of the human trainer and audience
84
self-fulfilling prophecies
behaving so that what you expect will become true
85
pygmalion effect
one’s expectations become reality like pygmalion’s sculpture coming into life
golem effect- negative effects (low expectations lead to poor performance)
86
rats in the maze example
self fulfilling prophecy
- rats that were expected to be good actually performed better
87
secondary observer effect
the researchers select and handle the data in a subjective and bias
happen during data sampling, recording , analyses and interpretation
biases due to secondary observer effects:
sampling bias
observer bias
selection bias
88
observer bias
systematic divergence from accurate facts during observation and recording of data
- confounders distort data
- researchers observe and record information differently
types of observer bias:
cognitive biases
detection biases
89
cognitive biases
researchers cognitive bias distort data collection
types of cognitive bias include confirmation buas and halo effect
confirmation bias :tendency to search for, interpret, favour and recall information in a way that confirms their beliefs or hypothesis
halo effect:
tendency for the positive impressions and beliefs in one area to influence a researchers data recording or interpretation in other unrelated areas
detection biases:
focusing on some cases to the detriment of others (e.g. checking diabetes in obese people and not other patients
detection biases can involve cognitive biases (the researcher’s focus) and or errors in sampling
90
control of investigator effects
1. minimise
2. assess
3. control
91
minimise investigator effects
double blind procedures
- ppt and researcher are unaware of allocation to experimental condition and cannot anticipate any result
^ counteracts demand characteristics and primary observer effects
92
assess investigator effects
observer reliability
- indicates how consistent or reliable measurements are
- inter-observer reliability - across different observers
- intra-observer reliability - within observer (retest-reliability
93
account for investigator effects
averaging across several researchers compensates for individual idiosyncrasies and biases, like when using a jury for ratings
94
summary link to validity
accuracy and reliability:
- systematic and unsystematic errors
internal validity
- measurements do not measure what they are meant to measure e.g. placebo effects
external validity
- results are not reproducible without experiment specific conditions
95
representativity of measurements (representativeness)
are measures of the experiment representative of the conditions you want to understand (external validity)
96
statistical population
the entire pool (or set) of people, items or events ability which a researcher wants to gain insight and draw conclusions in a study
97
statistical sample
a set of individuals or objects collected or selected from a statistical population by a defined procedure
98
sampling
the selection of a subset of individuals from within a statistical population to estimate characteristics of the whole population
99
representativity of a sample
complete sample:
- all members of the population, usually impossible to measure
representative sample:
- represents all relevant properties of the whole population so we can draw conclusions about the population based on measurements
- statical technique allows us to estimate the situation for the whole population based on the measured obtained from representative samples
100
sampling bias
- systematic error (bias) in sampling causing members of the intended population to be less likely be included in the sample
- undermined external validity - generalisation about the population
101
sampling bias
- inclusion in measurements
- miss important subpopulations in sample
- in process of gathering sample
- contradicts assumptions and research question
- distorts data and conclusions
- mainly external validity - undermines generalisation of findings beyond context of study
102
selection bias
- inclusion in statistical analysis
-discard outliers that are
103
selection bias
- inclusion in statistical analyses
- discard outliers which are meaningful
- after measurements (analysis)
- contradicts experimental design (control and manipulation)
- distorts statistical results and conclusions (data is ok)
- mainly internal - undermines interpretation of results within context of study (confound)
104
types of causal influence
necessary cause
sufficient cause
contributory cause
105
necessary cause
the cause is necessary to produce the effect (but might not be sufficient)
bananas must be ripe to taste good, but not all ripe bananas taste good
106
sufficient cause
the cause alone produces the effect, but it might not be necessary
bananas from kenya always taste good (but there are other good bananas)
107
contributory cause
the cause contributes to an effect by increasing its likelihood or strength
chocolate sauce increases the flavour of bananas, but if they are not ripe they are still not good, and there are ripe bananas that taste good even without chocolate
108
standard deviation
the square root of the average of each scores squared deviation from the mean score
the square root of variance
bigger value is more spread out
109
probability
ranges from 0-1
p(event A) = A/(A+B)
p(event A doesn’t occur) = B/(A+B)
110
mutually exclusive
occurrence of one event precludes the occurrence of another event
111
independent events
occurrence of one event has no effect on the probability of the occurrence of another event
112
additive rule for mutually exclusive events
the probability of occurrence of one event or another is equal to the sum of their separate probabilities
p(A or B) = p(A) +p(B)
clue OR
113
multiplicative rule for independent events
p(A and b) = p(A) x p(B)
clue AND
114
rules of and vs or
AND- multiply probabilities
OR- add their probabilities
115
addictive rule when not mutually exclusive
p(A or B) = p(A)+p(B)-p(A and B)
e.g. when picking a jack or a diamond have to take out the card which is both a jack and a diamond
116
multiplicative rule for non-independent events
p(A and B)= p(A) x p(B|A)
conditional probability - probability of one evnt given the occurrence of another p(B|A)
117
combinations
number of ways to arrange a subset of objects
NCr =(N!)/(r!(N-r)!
N is number of events
r is number of successes
! is multiply by all smaller integers
118
binomial distribution formula
calculates combinations and probability for any one combination of getting r out of N
when b number of events (N) is large the binomial distribution is equal to the normal distribution
119
binomial distribution formula
p(r) = N!/(r!(N-r)! p^r (1-p)^(N-r)
120
three ways of presenting results
text- verbal description - establish unambiguous logical relationships, could get lost in details
table- spatially organised representation of single, precise specifications - communicate large number of details, could get lost in details, arguments are not communicated
Graphics- visualisation of trends - focus and highlight main patterns of data, communicate different types of data - specific details e.g. decimals get lost - visual interpretation depends on viewer
121
data tables
organise information
communicate details
122
why use graphics?
Assess the quality of data:
- detect faults in experiment (bugs, incorrect data recording, misleading instructions, task too difficult)
- characterise data distribution (normal distribution, skew, kutosis)
- identify outliers (single data points outside the distribution that can be misleading
visualise results:
- highlight main results (test of main prediction)
- explain main results (reveal origin of results, expose spurious results, confounds)
123
How to make graphics?
1 Dimension - List, rank, line - no gain of visualisation
2D- Area - visualise patterns
3D - Relief, animation, interactive graphic - visualise complex relationships - difficult to interpret
4D - animation, interactive graphic - visualised complex relationships, difficult to interpret
124
how to use graphical tools?
1. how many variables
2. what types of variation
3. what scale
4. what resolution
5. what pattern or relationship
125
number of variables
One variable (univariate)
- data quality
- distribution
- outliers
two variables (bivariate)
- independent and dependent variable
- simple relationship
more than two variables (multivariate)
- two or more factors
- complex relationships
126
types of variation
independent variation - input to the empirical test, manipulated by experimenter, fixed in design - conditions of observation or measurements
Dependent variation - output of the empirical test depends on measurements and is expected to depend on the IV - observation/measurement
other variation (noise) - not part of testing logic, interferes with the tested variation and defines reliability of results
127
where do the variation go on the graph
IV - X axis
DV - Y axis
Noise - error bars
128
types of scales
Nominal, ordinal, interval and ratio
129
resolution
least resolution:
Binary data
discrete sets
discrete categories
continuous numbers
full description
Most resolution
130
decrease the resolution of data to avoid clutter
Nominal - categorisation
Quantitative - Aggregation and binning
131
Patterns
trends and relationships
132
Pie chart
Variable 1: continuous ratio-scaled circle (pie)
Variable 2: discrete data in sectors
+ relates sectors to whole circle
- limited to 2 sources of variation
- no error bars
- requires ratio scales but finite data to add up and define beginning and end of circumference
133
Bar chart
Variable 1: discrete x-axis
variable 2: quantitative scale along y axis
noise - error bars
additional discrete variables: grouping and/or visual appearance
+ Grouping
+ Highlight distance from zero
+highlight difference across groups or conditions
+ visualise different groups and conditions
- no continuous X
- useless for continuous data on both axes
- risks clutter when too many data points
134
Line chart
Variable 1: continuous x axis, at least ordinal scale
variable 2: continuous y axis, at least ordinal
noise - error bars along y-axis
additional continuous variables: grouping of data on sperate lines as identified by visual appearance
+ functions and tends - highlight relationship between values along the x-axis
- covariation
- useless when multiple data points per x-value (non-bijective data)
- less efficient for grouping than bars
135
scatter chart
variable 1 + 2: continuous at least interval
noise error bars possible along both axes
additional variables - colour (discouraged)
+ covariation
+ bivariate distribution
- lacks structure
- useless for discrete data - occlusion
- free-floating points
- high risk of clutter for 2+ variables
136
aggregation by frequencies
1. aggregation by counting each value
2. summing up a binary value
3. display in a frequency table
137
frequency table
proportion: a part, share or number considered in comparative relation to a whole - relative frequency
percentage = relative frequency * 100
138
What graphs to use for a nominal scale
Pie, Bar
139
What graphs to use for an ordinal scale
Bar
140
continuous data
Binning- transform continuous data into discrete data by allocating the continuous data to intervals, the bins
e.g. 140-150 = 2
150-160 = 5
this can be displayed in a stem and leaf diagram
which when turned on its side shows a histogram
141
what graph should be used for continuous data?
Bar (histogram)
142
histograms
symmetrical: no tails of data/2 tails
negative skew: left tail
positive skew: right tail
frequency density: frequency of data per equal interval = frequency / bin size
probability density: probability of data per value
143
continuous data - cumulative frequency
cumulative sum: summing up progressively
(adding up all the previous frequencies to the next)
cumulative histogram: x axis: binned height, y-axis: cumulative frequency
purpose of cumulative histogram:
cumulative probabilities- cumulative frequencies are an estimate of cumulative probabilities
144
what graph for a comparison of frequencies?
Bar graph
error bars with standard deviation are redundant in this case
standard deviation of binary variable only depends on proportions
square root (p*(1-p))
standard error:
square root (p*(1-p)/n)
145
aggregation by central tendency
standard deviation and mean
the mean is the bar and the standard deviation is the error bar
only representative for symmetrical distributions
146
asymmetrical distibutions
do a boxplot (box-and-whisker plot)
Median, quartiles, IQR shown
whiskers - 1.5*IQR
- the minimum and maximum of the data after excluding outliers
147
what graph should be used for a discrete IV?
Line graph - highlights trends
148
what graph should be used for a continuous IV?
Line graph
149
what graph should be used for a bivariate distribution?
scatter graph
150
Alhazen
Advocated observation to overcome subjectivity
151
Bacon
Baconian method - systematic tests of ideas (hypothesis) through observations
152
what did Alhazen and Bacon believe in?
Induction
153
critical rationalism
rejection of induction - general propositions can never be proven e.g. all bananas are yellow
154
critical rationalism: falsifiability
general propositions can be refuted = falsified
there is a red banana so not all bananas are yellow
155
criterion for scientific propositions
- testable = verified
- scope (is it s general insight)
- falsified (be refuted)
all scientific propositions (assumptions, hypothesis and conclusions) must be falsifiable
156
hypothetico-deductive model of scientific method
1. research question
2. hypothesis: general statement or explanation that answers the research question and can be tested (falsifiable)
3. predictions: deduce specific predictions from the hypothesis
4. test: falsify (disprove the hypothesis)
between 2 and 3 = deduction
logical error (affirming the consequent) to consider a confirmation of the specific prediction as a positive proof of the general hypothesis
157
operationalisation
translate general hypothesis into a specific prediction of the measurements obtained with your empirical test. the test defines the variables that you measure
hypothesis - operationalisation - prediction
by operationalising deduction is occurring
158
scientific process
Karl Popper
scientific progress is based on challenging existing general propositions (assumptions) proposing new ones (hypotheses) which will then be challenged again, and so on
159
scientific progress cycle
theory (rationalism)
deduction
empirical test
falsification
160
generalisation
translate specific results of a test into a general statement that contributes to a theory
results - generalisation - conclusion
induction occurs at generalisation
161
discussion
consider existing evidence and evaluate the contribution of your results to the general hypothesis
results - interpretation - discussion - generalisation - conclusion
162
limitations to the hypothetico-deductive model
- assumes absolute unidirectional relationships
- criticism of indoor ornithology
- research is often more complex
163
uncertainty of premise
the premise may be uncertain e.g is it covid or the flu
164
probabilistic relationships
the conclusion maybe about statistical likelihood, rather than a black-or-white (dichotomous) decision
e.g. swans are not necessarily white, but they are likely to be white
165
Bidirectional relationships
the conclusion may be about how A depends on B, and how B depends on A.
how likely is it that a swan is white and that a white animal is a swan
either: identify swans or exclude non-swans
whiteness is neither necessary nor sufficient for animal to be a swan
166
Bidirectional relationship: sensitivity and specificity
95% sensitivity - false negatives - 5% of cases are not detected
95% specificity - false positives - 5% of negative cases are falsely declared as positive
167
descriptive propositions
cooccurrence relationships: If A happens (antecedent), B (consequent) also happens. no order in time. no change or variation of B depending on A
168
explanatory propositions
casual relationships
- Effect happens due to cause
- Cause contributes to the state of effect - change and variation of effect depending on cause
169
cooccurrence and causation
causation is difficult to determine, can only observe cooccurrences (observe B when A happens)
cooccurrence doesn't imply causation
cooccurrence without causation is a spurious relationship
170
causation in research
research aims to show causal relationships.
can only observe cooccurrences
171
experiment
experimental manipulation - test how measurements change with and without the experimental condition
experimental control: keep all conditions the same with and without experimental manipulation
172
experimental operationalisation
- translate general hypothesis into variables (deduction)
- specifies the operation that results in the production of an outcome
- defines a DV in how its measured and IV in how it is controlled
173
experimental manipulation
- IV: variation across experimental conditions
- DV: variation of the measurements depending on the experimental conditions
174
experimental contol
confounder: condition/factor whose variation systematically affects the DV but which is not part of the conditions IV
noise: unsystematic random variation of measurements
175
experimental control: confounder
confounding = mixing up cause and effect
confounders produce cooccurrence without causation = spurious relationships
176
example of noise
grade varies independent of sleeping time
e.g. 4 different grades when students sleep 7 hours a night
noise does not produce spurious effects but ambiguity (uncertainty)
177
how to control confounds
keep all conditions CONSTANT that are not part of the experimental test
the core of experimentation: it allows identifying effects on the DV that only occur due to the variation of the IV
178
Control random noise
optimal: measurements of the DV are constant within a specific experimental condition
depends on control of conditions (IV) and control of measuring DV
179
challenges to confounders
keep all confounding conditions constant:
- confounding conditions might escape your control
some confounding conditions might only be similar, but not exactly the same.
180
challenges to noise
always some degree of noise in measurements even under optimal conditions
depending on other factors your measurements might not be as precise as they would under optimal conditions
181
continuum of research methods
Least control
Observations
Quasi
True experiments
most control
182
observational research
types: case studies, surveys, interviews, focus groups
conditions of measurements: situation and context choice, systematic organisation and structure of observations
no manipulation
measurement: qual or quant
183
weaknesses of observational research
no test of causality
- no experimental manipulation and control
- only cooccurrence
- causality fully depends on interpretation by researcher
imprecision of DV
- uncontrolled conditions may interfere with measurement e.g walk in front of object being observed
- clutter: time-consuming, overflow of data
184
true experiment
full control of IV and confounders
e.g. colour perception
everything constant
- motivation, concentration and fatigue can't be controlled
185
quasi-experiment
operational IV and DV
non-equivalent conditions/groups:
systematic differences between conditions/groups due to:
- IV not fully independent (random assignments in true experiments)
- IV is not controlled and manipulated
186
Quasi weaknesses
confounder in group comparison- systematic differences between groups may account for observed effects
sampling error or bias- samples not representative of population
confounder in experimental conditions - uncontrolled variation of experimental conditions can produce systematic or random variation of the DV
187
Quasi purpose
necessary in field studies because not possible or not ethical to manipulate IV
188
Types of quasi
- controlled but non-equivalent determination
- natural determination = natural experiments, IV is measured, but not fully controlled
189
reliability
reliability of:
1. Research procedures: information necessary to reproduce the same conditions of a study
2. reliability of results:
- test reliability
- statistical reliability
- experimental reliability
190
test-retest reliability
how reliable is your measurement instrument
low test-retest reliability: measurement is broken
191
statistical relaibility
how much noise is in your data?
conditions vary too much
192
experimental reliability
stable results across experiments
1. reproducibility
2. repeatability
3. replicability
193
reproducibility
other researchers in another lap reproduce the results with the original data and analyses
quality and transparency of data analysis
194
repeatability
results can be reproduced by the same researchers in the same lab by simply repeating the experiment and data analysis
certifies statistical reliability
195
replicability
other researcher in another lab reproduce the same results by replicating the experimental conditions
196
construct validity
197
internal validity
1. validity of experimental control
- the data is not due to confounders e.g. spurious correlations
2. construct and content validity
- validity of operationalisation, measures the idea or concept (construct) for which we use it and the extent to which it represents that construct
198
external validity
the extent to which the results of a study can be generalised so that conclusions apply beyond the context of the study
lack of external validity means that IV miss important causes
199
two types of external validity
1. ecological validity- result applies to different settings
2. population validity - representativeness - the result for a sample of participants applies to the whole population
200
relationships between different types of validity
accuracy and reliability are necessary but not sufficient for internal and external validity
valid instrument accuracy becomes equivalent to internal validity
internal validity contributes to external validity but could be a trade off as impossible to achieve both at same time
201
trade off between internal and external validity
not all factors may be known yet
effects of factors may not be understood so that they are confounders
202
lab study
artificial environment - control of IV and precise DV
+ full experimental control
+ maximise internal validity
- jeopardise external validity
203
field study
real environment and setting including all relevant factors. different types e.g ppt observation, field experiment
+ high external validity (ecological and population)
- reduced experimental control (internal validity)
- ethical issues
204
ppt observation
the researcher lives and participates with the informants
+ observation of maximal number of factors
- subjectivity of observation
- lack of experimental control
205
field experiment
can feature full experimental control or more likely a quasi-experiment
e.g. football shirts, Levine et al, (2005) ppts more likely to help an injured jogger wearing a football shirt from a team they supported - real life setting many confounders
206
what is noise?
random variation
- brightness of pixels an image vary randomly
207
what is a signal
systematic, regular = informative
noise = random = uncertain
208
bias
unwanted signal (systematic variation, systematic tendency) producing spurious results
in contrast to non-systematic random noise, conceals any kind of results
209
bias examples
scale: should show zero when nobody is on it, if not it is bias
handedness bias: answering with left and right finger in task - right handers yield better performance with right hand
210
2 types of biases
1. Independent bias- systematic errors - signals in measurements which are incorrect - limited to one variable (DV usually)
2. Dependent bias- confounder- produces bias in the DV varies with the IV
211
criteria of empirical evidence
noise reduces reliability and accuracy
systematic errors reduce accuracy
confounders reduce internal and external validity
212
sources of noise and bias
method (stimulus material and procedure)
participant (variation of physical condition, fatigue, cognitive processing, emotional responses)
Data recording- variation of experimenter or measurement instruments
213
minimise systematic errors
calibration of equipment - compare with reference measure
214
why control noise?
noise decreases the accuracy and reliability of measurements due to random variation. less likely to get the same results across measurements
aim of controlling noise: maximiser systematic variation due to causal effects relative to random variation (noise)
maximise the signal-to-noise-ratio
215
how to control noise?
Minimise or neutralise
1. minimise by using instruments with high precision
2. minimise by using a task with high precision
3. neutralise by sample size, if the noise is uniformly random increasing sample size makes the average converge to the expected value. noise isn't absent but neutralised
- increase number of ppts
- increase number of measurements per ppt
216
signal-to-noise-ratio
the amount of desired signal relative to the noise
increasing the ratio implies more information (reduction of uncertainty) of your measurement because noise is uncertainty
the higher the signal-to-noise-ratio, the higher the sensitivity and specificity of your measurements: less likely to get false negatives and false positives
217
noise and inferential statistics
how likely are you to detect an effect (signal, systematic variation) in a statistical test?
all test statistics express a SN Ratio
false positive - type 1 error - alpha - specificity
false negative - type 2 error - beta - sensitivity
sensitivity to detect effects - statistical power
218
why control confounders?
controlling confounders is fundamental for internal and external validity if results are due to a confounder
internal validity - conclusions are unwarranted - no validity
external validity - generalisation is misleading - no validity, results might turn out differently in real settings
219
how to control confounders?
1. fix
2. randomise
3. balance
4. measure and model
2 and 3 are to neutralise
220
how to fix confounders
keep all conditions constant that might potentially affect your measurements
when held constant, potential confounder is a control variable
fixing is not always possible - order effects - ppts characteristics will always vary across ppts - minimising is not sufficient as still systematic unwanted signal interfering
221
how to randomise to control confounders?
converts confounder (systematic variation) into noise (uniform random variation)
biases become orthogonal (uncorrelated) to the IV
can be neutralised by central tendency equal across conditions
limitations
- only works if sample size is sufficient
- only works if confounder affects all conditions in the same way
222
how to balance to control confounders?
dividing a set into two or more subsets that have roughly the same characteristics
- equalise central tendency across conditions of IV, confounder on DV is neutralised
systematic neuralisation 0 values of confounder are distributed in equal frequencies across conditions of IV, because values are symmetrical because central tendency is unbiased
limitations:
- only works if confounder is known before hand
- only work if confounder affects all conditions in the same way
223
how to fix confounders by measure and model?
measure potential confounder - it becomes a measured control variable (CV)
include CV in stats model when analysing data - test for interaction effects
confounder becomes an IV in stats model
results indicate whether or not there was a bias and how it affected the DV
stats model may account for effect making assumptions about the nature of the effect
224
why have a research protocol?
specifying details of experimental design in written format
fix experimental conditions across measurements and across experiments
maximise consistency across measurements - reliability
minimise noise and confounders produced by research procedure, increasing test and experimental reliability
225
content of a research proposal
operational definitions of conditions of IV, measurement of DV, experimental settings, such as lighting, distance from light...
instructions for experimenter- how to proceed and handle stimulus material
instructions for ppts - what to tell the ppts and how written/oral
method sections in scientific articles
226
typical structure of experiments
trial= one completion of a task, providing one datapoint for each DV
Block= trials with a specific characteristic concerning the control of confounders and manipulation of IV
Session= may feature blocks for each task
Run= experiment may involve several sessions
each ppt completes one run of the experiment, providing one raw dataset
227
manipulating IV
repeated measures design
between groups
228
repeated measures design
- same ppts in all conditions
- the IV is manipulated within ppts
- also called paired-sample when only two conditions
- two ways to organise sequences of repeated measures (blocked, interleaved)
229
Block design
green target block and blue target block
consequence: order effects due to:
- practice + learning
- anticipation
- fatigue
230
counterbalancing
- half of the ppts start with green, the others start with blue block
- to control for order effects
231
interleaved design
benefits
+ order effects controlled through randomisation across trials
+ no expectations towards condition of IV because ppts don't know what target to look for
problems
- noise due to unexpected changes
- bias when effects of sequence differ between experimental conditions
232
between groups design
A. Random groups:
- assign ppts randomly to each group/condition
B. matched-pair design - match ppts in characteristics that may be important, such as age
challenges of between groups design: potential noise and confounds because groups vary due to ppt characteristics, different approaches use randomisation and balancing
233
strengths and weaknesses of repeated measures designs
+ reduced noise and confounds by excluding differences between groups
+ wanted when practice effects are desirable
blocked design
- order and carry over effects (if no counterbalancing)
interleaved design
- produce noise due to task switching and mistakes
both
- interaction effects with trial order (practice, fatigue)
234
strengths and weaknesses between-group designs
between groups design
+ controls order and carry-over effects
+ wanted when inexperienced, naïve ppts needed
random groups
- large ppt samples needs to allow for neutralising by randomisation
matched group:
- failure to account for ppt variation
- vulnerable to dropouts
235
Factorial design
multiple factors (IV's)
multiple independent variables
manipulation of IV's:
- between ppts
- within ppts
- both between and within ppts
mixed factorial designs:
- both between and within factors are manipulated together
- very common approach
236
interaction effect
interaction between factors:
= effect of one factor depends on another factor
= multiplicative effects
quantitative = in the same direction for all conditions (change only in magnitude)
qualitative = in opposite directions some positive effects others negative (effects change direction)
interaction: the difference before and after depends on the type of therapy
quantitative interaction: both therapies improve happiness, but therapy 1 improves more than therapy 2
237
factorial design advantages
internal validity:
+ more than one hypothesis can be tested because you are manipulating multiple IV
+ allows for assessing complex casual relationship, interaction effects
+ allows for including potential confounders in stats model
ecological validity
+ more complex designs can be more relevant to the real world
238
multivariate design
multivariate = more than one DV (effect of walking on physical and mental health)
involves relationships between IV and DV
239
historical scandals in ethics
Little Albert
Monster study
Milgram
Zimbardo
Harlow
240
Little Albert
9mth orphan
conditioned albert to be fearful of rats by shouting and making loud noises
- no attempts to conditioning afterwards
- potential negative consequences on development
241
monster study
22 children from Iowa soldiers orphans
half received negative treatment e.g. belittling the children for speech imperfection
- no consent, no information, no debriefing and no help afterwards
- negative psychological effects and speech problems for rest of life
242
milgram
- ppts gave consent but were deceived
- no debrief
- inflicted insight without being explained that his was part of the experiment
- ppts did not feel well during the experiment
243
zimbardo
- stimulation of a prison environment where ppts took the role of prisoners or guards
- study authority effects behaviour
criticisms
- did not obtain consent
- ppt suffered psychologically and physically during experiment
- hindered ppts to withdraw
- debriefing too late, leaving ppts under the impression of the experiment
244
Harlow
isolation of infant and juvenile macaques
depression and social deprivation link
- unethical treatment of monkeys during experiment
- effect of monkeys beyond experiment
245
the belmont report
respect for people:
- informed consent
- voluntary ppt
- no/minimal deception
- anonymised data
- confidentiality
Beneficence
- ppt welfare: beneficial, not harmful
justice
- fair selection of ppts
246
ethical guidelines
BPS
principles: respect, scientific value, social responsibility, maximising benefit and minimising harm
risk assessment, informed consent, confidentiality, advice, justify deception, debrief and GDPR
247
informed consent
A) consent - ppt voluntarily confirms willingness to participate in study
B) ppt first needs all information before deciding whether to participate
- ppt information sheet, consent form
248
debrief
purpose: ppts are fully informed about the study and not harmed in the process
especially important in social psychology that use deception
format - short interview, written debriefing statement
249
review
application to ethics committee ethics committee checks:
risk assessment
information sheet
debriefing statement
consent form
debriefing statement
general information about your study
posters and adverts
recruitment methods
250
agency
human beings act upon and shape their environment
251
interpretation
people perceive think and make choices to adapt their actions and behaviour
252
double interpretation
one person interprets how another person interprets and acts accordingly which may in turn influence the interpretation and action of the other person
253
social construction
constructed understandings of the world form the basis for shared assumptions about reality
based on mutual expectations
unconscious and automatic
254
double hermeneutic
from sociology
ppt interpret the researchers behaviour and the setting of the study and adapt their behaviour, which is then interpreted by the researcher
shapes and observes reality
255
sampling methoods
define population
identify sampling frame
choose sampling method
256
inclusion and exclusion criteria
depends on:
- population of research question
- prior knowledge and assumptions
257
sampling frame
depends on resource
sampling frame can nevertheless bias research
258
types of sampling methods
non-probability sampling
probability sampling
259
non- probability sample
self-selecting / volunteer samples
opportunity / convenience samples
snowball samples - one ppt from a key group gives leads to others from this group
260
sampling biases in non-probability samples
exclusion bias: exclusion of particular groups
self-selection bias: when ppt can decide whether to participate, people with specific characteristics are more likely to agree to take part in a study with others
261
sampling biases in nonprobability samples
WEIRD ppts
Wester, Educated, Industrialised, Rich and Democratic sample frame
262
probability sampling
convert systematic biases to noise through randomisation to obtains representative unbiased sample
Every unit in the population has a known nonzero probability of being sampled
random selection of units by weighting units according to their probability of selection
random sampling
systematic sampling
stratified sampling
263
random sampling
each element of the frame has an equal probability of selection
the sampling frame is not subdivided or partitioned
all subsets of a sampling frame have an equal probability of being selected
264
systematic sampling
also quasi-random sampling
starting from a random, every nth person from a list is selected
265
266
stratified sampling
stratum = sub-population
stratification = diving members of the population into homogenous subgroups before sampling
each stratum is sampled randomly as an independent sub-population
the number of individuals per stratum is proportional to the size of strata in the population
combines balancing and randomisation in ppt sampling
267
representativity and validity
population validity= is the sample representative
ecological validity= is the stimulus representative and the task representative
experimental control - internal validity
representativity - external validity
268
observational studies and subtypes
observational studies = IV is not under experimental control - correlation studies
subtypes:
Comparative research: comparisons between countries and cultures
Case-control study: comparing subjects who have that condition with patients who do not but are otherwise similar (the controls)
Case study: in depth detailed examination of a particular case
Longitudinal study: repeated observations of the same variables over short or long periods of time
269
review and meta-analysis
review
summarises the current state of understanding on a topic or research question
uses mainly logical argument to integrate existing evidence
meta-analysis
aimed at drawing conclusions about evidence across existing studies
uses statistical analysis to combine the results of multiple exisiting empirical studies
270
communication and dissemination
research report:
- scientific articles
- other publications
conference presentations
meetings
- invited speakers (colloquia, seminars)
- workshops, symposia
- informal chats and discussions
collaborations
271
research report and scientific articles
introduction
- assumptions
- research question
- hypothesis
- prediction
method
- research design
result
- data analysis
- interpretation
discussion
- internal and external validity
- generalisation
conclusion
272
interests
financial & material resources:
= research funding
includes:
- salary
- lab equipment and research expenses
- financial support for research institution
standing and reputation
- awards, invitations, professional roles and influence on decisions
- within institution
- across international scientific community
- across society
273
currency of research: publication record
strong publication records to:
- get a job
- get research funding
- gain wider acceptance from scientific community
strong publication record means:
- many papers = good
- papers in prestigious, high profile journals = good
274
problems: conflict of interest
journals can refuse to publish research that presents null results - failed experiment
strong incentive to obtain significant results from experiments - no incentive to obtain null results
bolder claims - higher risk to fail
275
personal interest vs scientific quality
e.g Brian Wansink
eating behaviour
he manipulated data to fit theories
his article has been the subject of retractions
276
concequences
discipline in crisis
- failed replications of studies so researchers have conducted experiments wrong?
discipline in progress
- better checks of data, method and conclusions
- open science: make all details including data available and transparent
277
quality control
1. resources
- financed by a third party (usually tax payers)
- should not be wasted with useless research
2. ethics
- participants must not be endangered or harmed
3. dissemination
- findings are sound and conclusive
- otherwise: misleading, fake news!
278
quality control via peer review
279
data distribution
the distribution shape is the curve enclosing a histogram
280
normal distribution
assume everything follows a normal distribution. symmetrical and bell-shaped.
281
standard deviations affect the distribution
wider peak
282
symmetry in distribution
mean=median=mode
283
asymmetric data is skewed
range -infinity to +infinity
skew is greater than +/-2 the data is substantially skewed
direction of tail = direction of skew
negative skew- left tail
positive skew- right tail
284
positive skew
mean > median > mode
285
negative skew
mean < median < mode
286
kutosis
peakedness (altitude) of this distribution
relative concentration of scores in the centre
-2 (flat) to +infinity (peaked)
mesokurtic (normal distribution)
platykurtic (flat, thick in the shoulders, wide peak)
leptokurtic (peaked, thick in the centre and tails, narrow peak)
multimodal- more than one peak
if the kurtosis is greater than 1.96, then the distribution of the data is significantly different from mesokurtic
287
the normal distribution
bell shape distribution
skewness and kurtosis are 0
more stats tests assume data is normally distributed
data can be transformed to meet the assumption
288
transformation
apply a mathematical transformation to each score e.g. multiply it by itself (square it)
289
non-linear transformations to a positive skew data
moderate skew: square root X
Substantial skew: log(X)
severe skew: reciprocal transformation (1/X)
290
non-linear transformation to negative skew
reflect the distribution (multiply by -1)
add a constant so the lowest value is 1.0
moderate skew square root X
substantial skew log(X)
severe skew reciprocal (1/X for each value)
291
do nonlinear transformations change the shape of the distribution of scores?
YES!
292
do linear transformations change the shape of distribution of scores?
NO!
293
linear transformations
add, subtract, multiply or divide
distribution shape remains the same
e.g. Celsius to Fahrenheit
294
link between the distribution of data and probability of obtaining a particular value
tabling the distribution can give estimates of probability
295
Normal distribution
bell shaped (symmetrical)
standard normal distribution
mean= 0
standard deviation - 1
possible to transform any normal distribution
296
transforming distributions
if we subtract a constant from each score, the mean of the distribution is reduced by that constant (subtracting the mean from all values in the distribution gives us a mean of 0)
when divide all values by a constant, we divide the SD by that constant giving us a SD of 1
297
z-scores
z-score tells you how many SD units a particular score is from the mean
scores range from -infinity to +infinity although most are in the range of +/- 2
formula= (x-mean)/standard deviation
298
z-scores
mean + standard deviation
mean - standard deviation
the answers to the formulas above give the extreme values so anything above or below are extreme
299
95% of scores are within how many standard deviations of the mean
± 1.96
300
the standard normal distribution, z-scores
mean = 0 and standard deviation = 1
b values greater than 1.96 or less than 1.96 are extreme (95%)
301
what does a z-score tell you?
z-score indicates that the observation is greater or less than the mean
z-score magnitude tells you how far the score is from the mean
302
how to calculate the z-score?
(score-mean of scores) / standard deviation
303
how to calculate that z-scores are the same distance from 0 will have the same chance of occurring
probability of several shaded areas = sum of probability of the shaded areas
probability of non shaded areas = 1 - probability of shaded areas
304
z-score tables
show the percentage above a certain z-score so if want to know the amount less than the score do 100 - percentage
305
dealing with outliers
outliers are extreme scores
correcting outliers
- check original data
- remove from dataset
- Change the score
replace with the next highest score plus one
replace with mean plus or minus 2 x SDonly if it remains the highest or lowest score
95% o the data lie between 2 SD of the mean
306
what does inferential statistics do?
infer from sample to population
measure statistics of samples
infer parameters of population
307
sampling distribution
sample mean is unbiased estimator of the population mean, but rarely will it be the same because of error
mean of all possible sample means is equal to the population mean
308
central limit theory:
1. the sampling distribution of the mean will have a mean equal to population mean (all possible sample means)
sample mean usually ≠ population mean
309
sampling distribution
the standard deviation of the sampling distribution of the mean is called the standard error of the mean
310
standard error
measure of the amount that a sample mean could be different from the population mean
