Research Methods Flashcards
the belmond report 3 principles
- respect: informed consent, special groups protection
- justice: right balance between the people that benefit from the research and the people that participate in it
- beneficience: participants should be protected from harm and ensured well being. ook confidentiality hoort hierbij.
APA 5 general principles
- respect
- beneficience
- justice
- fidelity and responsibility (trust, professional)
- integrity (accurate info, honest)
deception through omission
witholding info
deception through comission
lying
difference data fabrication and data falsification
- Data fabrication: Inventing data to fit the hypothesis
- Data falsification: Influencing a study’s results
3Rs from animal research
replacement (liever een andere methode), refinement (minste distress), reduction (zo min als mogelijk dieren)
interaction effects
when the original independent variable depends on the level of another independent variable
spreading interaction
an interaction between two independent variables “only occurs when..”
moderator
independent variable that changes the relationship between the independent and dependent variable
factorial design 2 effects
main and interaction effects
interaction effects wanneer ?
als de lijnen parallel zijn: GEEN interactie effect. anders wel.
factorial variations
- Independent-groups factorial designs: Both independent variables are studies as independent groups (E.g.: in a 2×2 factorial design, there are four different groups of participants in the experiment)
- Within-groups factorial designs: Both independent variables are manipulated as within-groups (E.g.: in a 2×2 factorial design, there is only one group of participants, but they participate in all four combinations of the design)
- Mixed factorial designs: One independent variable is manipulated as independent-groups, and the other is manipulated as within-groups
three way design
- Results in three possible main effects
- Results in three possible two-way interactions
three way interaction
Three-way-interactions: A two-way interaction that depends on the level of the third independent variable
obscuring factor
hierdoor kan je geen relatie zien tussen independent and dependent variable
internal validity
zijn er andere factoren die een rol spelen?
design confound
andere explanation omdat het studie design gewoon niet goed is
selection effects
the participant groups are systematically different
order effects
the order in which the interventions are presented lead to differences: participants get tired etc.
maturation threat
spontaneous remission (behaviour just changes)
history threat
a factor unrelated to the study influences the outcome for the whole group (COVID)
regression to the mean
when a group has an extreme value as mean, it is most definitely going to be less the next time.
attrition threat
drop out
testing threat
participants scores change when they take a test multiple times
instrumentation threat
the instrument changes -> measures differently
observer bias
researchers expectation have influence
demand characteristics
participant knows what the study is about
null effect
gewoon geen effect
between group obscuring factors
- weak manipulation -> door slechte study design geen goede manipulatie (manipulation check!)
- insensitive measures (door slechte measures kan je geen verschil meten tussen de groepen)
(- ceiling effect: all scores clustered at the end, te makkelijk) - floor effect: all scores clustered at the bottom, te moeilijk)
dus obscuring factors van verschillen zijn …
within group issues. de grafieken liggen dan verder uit elkaar.
within group issues of obscuring factors
- Noise: Unsystematic variability
- Measurement error: A human or instrument factor that can inflate or deflate a person’s true score on the dependent variable
- Individual differences (use within group design to solve this, or more participants)
- Situational noise: External distractions (control wat je kan controleren, anders randomiseren)
strengths of within groups
control for individual differences
fewer participants needed
strength of between group design with more than one independent variable
- two for the price of one
- you can study interaction effects
maar (-) wel 2x zoveel participanten nodig.
hoe kan je zien of er een 3 way interactie is
als de twee grafieken hetzelfde zijn voor verschillende locaties bijvoorbeeld -> dan geen interactie effect. als ze ander zijn is er wel een interactieeffect.
participant variables
variables that you do not manipulate. these are also called additional independent variables.
mixed factorial design
manipulate one independent variable between subjects and another within subjects.
For example, a researcher might choose to treat cell phone use as a within-subjects factor by testing the same participants both while using a cell phone and while not using a cell phone (while counterbalancing the order of these two conditions). But he or she might choose to treat time of day as a between-subjects factor by testing each participant either during the day or during the night (perhaps because this only requires them to come in for testing once). Thus each participant in this mixed design would be tested in two of the four conditions.
within-group design
pretest -> intervention -> posttest. in the same group
voorbeeld van regression to the mean
mensen die zich depressief voelen zullen zich de volgende keer waarschijnlijk minder depressief voelen, aangezien ze zich aanmelden voor therapie op hun laagste punt. daardoor wss beter de next measurement.
mixed factorial design
pretest -> intervention -> posttest
pretest ———————–> posttest
non-specific effects
effects not due to the treatment, but due to the expectation of being treated (bijvoorbeeld meer gaan sporten door motivatie therapie)
wat is een oplossing voor non-specific effects
mixed factorial design with control group on a waitlist
differences non-specific effects and placebo effect
non-specific: differences because you think you will be treated
placebo: because you are being treated
hoe doen ze vaak control condition bij psychological interventions
positive control group: kijken of de interventie beter is dan de traditionele interventie.
difference observer effect and observer bias
Observer bias: The researcher may see differences between the conditions that are not actually there
Observer effect: The researcher may treat participants differently depending on the condition they’re in
demand characteristics
The participant may have the tendency to behave according to the research hypothesis
hoe non-specific effects meten
door waiting list controls met controls te vergelijken
hoe meet je placebo effect
negative controls vs waitlist control
attrition
drop out, problematic if it is in one condition
testing
the effect of being tested twice (oplossing: control group of alleen posttest).
instrumentation
door instrumenten. oplossing: control group of alleen posttest.
small n reasons
- rare condition
- homogenous population (fridges)
- everyone has different symptoms -> everyone is a new experiment
- you need to know within-subject differences for psychological disorders.
small n vs large n
small n = differences between individuals. large n is differences within population
3 small n designs
- stable baseline design
- reversal design
- multiple baseline design
stable baseline design
[M1 M2 M3] T [M4 M5 M6]
stable baseline design waar controleer je voor en waar niet voor
- Solves the measurement problem
- Accurate estimate of individual parameter
- Controls for some internal validity threats
- Maturation
- History
- Non-specific effect
- Regression to the mean
niet: placebo of non-specific effects?
reversal design
you run consecutive sessions of the same condition until stable, and then you switch to another condition. if the behaviour differs between control and testing phase, the effect can be attributed to the treatment.
multiple baseline design
study the effect of the intervention on multiple variables.
when quasi experiment
if experiment is…
unethical
unpractical
unnatural
impossible
quasiexperiment kenmerken
geen randomisatie, en niet dubbelblind. maar een groep deelnemers die interventie ondergaat wordt vergeleken met een groep die geen interventie ondergaat. the manipulation is an already scheduled event, therefore the researcher is not in control (bv plastische chirurgie).
power =
rejecting the H0 when it is indeed not the correct hypothesis.
difference substantive and statistical hypothesis
sustantive: hypothesis about how the world works.
statistical: statement about a population parameter
reject H0 if..
your data is not likely if H0 is true
not reject H0 if
your data is likely if H0 is true
p-value
probability of these or more extreme results if H0 is true
test statistic
how far the point estimate falls from the parameter value in the null hypothesis
p-value bij two tailed test
the probability on these or extremER results if the H0 is true
proportion assumptions
- Variable is categorical
- Data are obtained using randomization (random sample or random assignment)
- Sampling distribution of proportion is normal if np ≥ 15 and n(1-p) ≥ 15
p-value bij <
gewoon zo
p-value bij >
1- …
*
Conclusions
from a two sided significance test will agree with
conclusions drawn from a confidence interval
*
e.g.,: a significance test with a significance level of 0.05 will produce
the same conclusions as a 95% confidence interval
*
e.g.,: a
significance test with a significance level of 0.01 will produce
the same conclusions as a 99% confidence interval
oke
type 1 error
H0 gets rejected when it is right
type II error
H0 does not get rejected when it is false
power =
1- type 2 error
wat doe je bij het vergelijken van 2 verschillende groepen obv means
H0 : u1 - u2 = 0
Ha: u1-u2 =/= 0
standard error =
=s = the estimated standard deviation of the sampling distribution
df with two independent groups
is we can assume that s1=s2; then df = n1 + n2 - 2
hoe t value krijgen uit df
t.inv
95% CI bij twee groepen =
x1 - x2 +- t0,025*se
a 95% CI means that …
that in the long run,
95% of your confidence intervals will include the true parameter value
observational study
compare the results of one group to a standard
a sampling distribution for 2 proportions is normal when….
n 1 p 1 ≥ 10 and n 1 (1 p 1 ) ≥ 10 in group 1 and if n 2 p 2 ≥ 10 and n 2 (1 p 2 ) ≥ 10 in group 2
a sampling distribution for 2 proportions is normal when….
n 1 p 1 ≥ 10 and n 1 (1 p 1 ) ≥ 10 in group 1 and if n 2 p 2 ≥ 10 and n 2 (1 p 2 ) ≥ 10 in group 2
dus het verschil tussen 1 proportion en 2 proportions between independent groups:
de assumptie over normale verdeling:
1 proportion = normally distributed if np > 15
2 proportions = normally distributed if n1p1 > 10
(3 for two sided test: n*p >5)
(two proportions has a smaller necessary assumption than 1)
if the interval contains 0…
the proportions/means do not differ significantly
assumptions for two proportions in 2 sided test
n*p => 5!
hoe bereken je ^p voor 2 proporties
= overall passed/n1 + n2
𝑥̅𝑑 berekenen: in welke sample en hoe?
in matche-pairs of repeated measures.
alle scores van elkaar aftrekken en daar het gemiddelde van nemen (x1 - x2 etc).
wat is n bij dependent samples
aantal paren!!!!!!!!!!!!!!!!!!!!!!!!
wanneer mcnemar’s test
bij dependent samples: proportions
assumptions of mc nemars test
- categorical variables
- the sum of the frequencies is at least 30
fomule -> z score -> p value
wat te doen bij 2 categorical variables…
contingency table!
conditional proportion
the proportion of the response variable, for one level of the explanatory variable
what is independence for conditional proportions
independence: a response variable is independent of the explanatory variable (no association!) if the conditional proportions are the same.
what is dependence for conditional proportions
dependence: a response variable is dependent of the explanatory variable (association) if the conditional proportions are different
assumptions for when both variables are categorical: strength of assocication
- categorical
- random
- each cell has 5 expected counts
hypotheses for 2 categorical variables: strength of association
H0: the variables are independent
Ha: the variables are dependent
what does the H0 of cat ass imply?
that P(A and B) = P(A) * P(B) (want multiplication rule for independent H0!) events.
If H0 is true, then P(A and B) = P(A) * P(B)
table maken expected by H0
- proportions normaal aan de RANDEN (deel/totaal!)
- middelste cellen: rand x rand
- alle getallen x n -> zodat je de expected frequencies hebt ipv proportions.
(assumption: if any of these variables is < 5, the test should not be performed).
dus….
expected frequency = (row total x column total)/total n
chi square test voor…
determining how much the observed and expected values differentiate from each other.
x2 distribution
- always positive
- df = (rows -1) *(columns -1)
- mean = df
- two sided always
- large x2 = evidence against independence = H0 is rejected (large x2 = association likely)
hoe van p value naar x2 value
chisq.inv(0,95;df)
hoe van x2 naar p value
chisq.dist (voor links), 1-chisq.dist (voor rechts) -> automatisch voor two sided, dus nooit x2!
difference between x2 and statistical test
large x2 implies strong evidence against H0, pvalue says nothing about the size/strongness of association
two measures for strength of association
- difference of the conditional proportions between treatment and control group
- relative risk: ratio of conditional proportions
bij categorical, wanneer gebruik je wat?
2x2 rows: je kan z score gebruiken
meer dan 2 rows: chi square test gebruiken!
als je de z score gebruikt voor tabel, hoe doe je dit met de proporties berekenen?
^p = aantal successen/totaal n
^p1 = aantal successen ene groep / totaal n
^p2 = aantal sucessen andere groep / totaal n
x2 distribution is eigenlijk…
sampling distribution under H0
welke waarde vul je hier in bij de chisq.inv(x;df)
probability = 0,95!!!! niet 0.05!!!!
hoe bereken je population proportions
deel/row total
HOE KOM JE BIJ DE T SCORE VOOR T0,025
t0,025 -> t.inv(0,975)!!!!
