Correlations and Experiments, Single-factor and factorial designs

Question 1

Pearson’s r

Answer 1

= quantification of a correlational relationship = product-moment correlation coefficient
-> indicates the degree of linear relation between 2 variables
-> varies between -1 and +1
-> absolute value indicates the strength
-> +/- indicates the direction of the correlation

Question 2

effect size

Answer 2

In statistics, the effect size is a measure that quantifies the strength or magnitude of the relationship between two variables or the magnitude of a difference between groups. It is used to provide a clear and meaningful understanding of the practical significance of a statistical result, as opposed to just the statistical significance. Effect size helps researchers and analysts determine whether an observed effect is large enough to be practically important or meaningful.

Question 3

significance level

Answer 3

A significance level, often denoted as α (alpha), is a threshold or critical value used in hypothesis testing in statistics. It is a predetermined level of probability that is used to determine whether a statistical result is considered statistically significant or not. In other words, the significance level helps in making decisions about the null hypothesis.

Setting the Significance Level: Before conducting a statistical test, researchers choose a significance level, typically a small value such as 0.05 (5%) or 0.01 (1%). This value represents the maximum allowable probability of making a Type I error, which is the error of rejecting a true null hypothesis (a false positive).

Conducting the Test: After setting the significance level, researchers perform the statistical test, which typically involves comparing sample data to a null hypothesis. The test calculates a p-value, which is the probability of obtaining the observed results (or more extreme results) if the null hypothesis is true.

Making a Decision: If the p-value is less than or equal to the chosen significance level (α), it is considered statistically significant. In this case, researchers typically reject the null hypothesis and conclude that there is evidence to support the alternative hypothesis. If the p-value is greater than α, the results are not considered statistically significant, and researchers do not reject the null hypothesis.

Question 4

directionality problem

Answer 4

direction of correlation is unknown

Question 5

third variable problem

Answer 5

a 3rd variable that affects both other variables is present

Question 6

criteria to establish causation

Answer 6

1. Covariance: results must show that variables correlate
2. Temporal precedence: method must establish which variable comes first in time
3. Internal validity: there must not be a C variable that is associated with variables A and B, independently

Question 7

cross-lagged panel procedure

Answer 7

The “cross-lagged panel procedure” is a statistical technique used in correlational research to examine the direction of causality between two or more variables over time. It involves analyzing multiple measurements of these variables taken at different points in time and looking for patterns or relationships that suggest the order in which they occur

Question 8

what if the correlation is 0?

Answer 8

• restriction of range (or truncated range)
• non-linear relationship
• no association

Question 9

experiment

Answer 9

when a articular comparison is produced while other aspects of the situation are held constant

Question 10

ceiling effect

Answer 10

test was too easy

Question 11

floor effect

Answer 11

test was too difficult

Question 12

reasons for Null results in experiments

Answer 12

• independent variable did not affect dependent variable
• manipulation was too weak
• insensitive measures
• ceiling effect
• floor effect
• unreliable measurement
• not enough power (small N)

Question 13

experimental designs

Answer 13

How to assign subjects to different levels of the independent variable:
1. Between-subjects design
2. Within-subjects design

Question 14

Between-subjects design

Answer 14

= independent-groups design
= separate groups of subjects receive different levels of the independent variables

Question 15

Within-subjects design

Answer 15

= repeated-measures design
= all subjects receive all levels of the independent variables

Question 16

Between-subjects design: advantages

Answer 16

there is no chance that one level of the independent variable affects performance under the other level of the independent variable

Question 17

Between-subjects design: disadvantages

Answer 17

there is a chance that the groups differ
-> can be “solved” by random assignment / matching

Question 18

Between-subjects design: when to use?

Answer 18

• the independent variable has a permanent effect
• large risk of order effects

Question 19

Within-subjects design: advantages

Answer 19

there is no chance of group differences. each subjects serves as their own control

Question 20

Within-subjects design: disadvantages

Answer 20

there is a chance of order effects (e.g. practice, carryover)
-> can be “solved” by counterbalancing methods or random order

Question 21

partial counterbalancing

Answer 21

Partial counterbalancing is used when it’s not feasible to present every possible order to each participant, which can be the case in experiments with a large number of conditions. In partial counterbalancing, a subset of possible orders is used, and this subset is typically determined based on a systematic approach. For example, if you have four conditions (A, B, C, D), you might use a partial counterbalancing scheme like ABDC, CADB, BCAD, DCBA. This method still helps control for order effects but is less comprehensive than full counterbalancing.

Question 22

Full counterbalancing / Complete counterbalancing

Answer 22

In a full counterbalancing design, every possible order or sequence of conditions is presented to each participant. This means that if you have, for example, three conditions (A, B, and C), each participant experiences all six possible sequences (ABC, ACB, BAC, BCA, CAB, CBA). This ensures that any order or sequence effects are equally distributed across participants, making it possible to isolate the effects of the independent variable from the order in which conditions are presented.

Question 23

Latin Square Design

Answer 23

A Latin square design is a structured approach to partial counterbalancing. It’s especially useful when you have more conditions than can be easily counterbalanced. In a Latin square design, you have as many rows and columns as you have conditions. Each condition appears once in each row and once in each column. This design ensures that each condition appears in each position an equal number of times. Latin squares are particularly useful when there is a concern about order effects and when full counterbalancing is impractical.

Question 24

Balanced Latin-Square Design

Answer 24

A balanced Latin square design takes the concept of a Latin square a step further. It ensures that not only do conditions appear equally often in each position but also that each condition appears equally often after every other condition. This design offers a more rigorous control over order effects.

Question 25

Within-subjects design: when to use?

Answer 25

• when the independent variable does not have a permanent effect and order effects can be avoided by counterbalancing or random order

Question 26

in general: within-subjects or between-subjects design?

Answer 26

in general, within-subjects designs are more efficient than between-subjects design (have more power)

Question 27

factorial design

Answer 27

experimental design in which each level of every independent variable occurs with all levels of the other independent variable
e.g. with a variable A and B and levels A1, A2, A3 and B1, B2 this would result in 6 different combinations

Question 28

between-subjects factorial design

Answer 28

A and B are both between-subject factors, e.g. we have 6 different groups

Question 29

within-subjects factorial design

Answer 29

A and B are both within-subject factors, e.g. we have 6 sequences (in Latin Square design)

Question 30

mixed factorial design

Answer 30

one of the factors (e.g. A) is a between-subjects factor, the other one (e.g. B) is a within-subjects factor
(in our example this would result in 3 groups doing 2 sequences)

Question 31

advantages of factorial designs

Answer 31

1. efficiency (more factors in 1 experiment)
2. generalization
3. information on interaction between factors

Question 32

main effect

Answer 32

= the overall effect of one independent variable on the dependent variable, averaged over the levels of the independent variable
Example: Imagine an experiment in which researchers are studying the effect of both “amount of sleep” and “caffeine intake” on cognitive performance. There are two independent variables:
* “Amount of Sleep” with two levels (6 hours and 8 hours).
* “Caffeine Intake” with two levels (low caffeine and high caffeine).
If the results show that cognitive performance is significantly better with 8 hours of sleep compared to 6 hours, this indicates a main effect of “Amount of Sleep.” It tells us that, on average, more sleep leads to better cognitive performance, regardless of caffeine intake.

Question 33

interaction effect

Answer 33

= when the effect of 1 independent variable depends on the level of the other independent variable(s)
= it represents a situation where the combined effect of 2 or more factors is not simply the sum of their individual effects
Example: Continuing with the same experiment, suppose that the results reveal an interaction effect between “Amount of Sleep” and “Caffeine Intake.” It means that the influence of sleep on cognitive performance is not the same for individuals with low caffeine intake as it is for those with high caffeine intake.
For instance, it could be the case that:
* With low caffeine intake, there is a significant difference in cognitive performance between 6 hours and 8 hours of sleep (indicating a main effect of “Amount of Sleep”).
* With high caffeine intake, there is no significant difference in cognitive performance between 6 hours and 8 hours of sleep (indicating a main effect of “Amount of Sleep”).
* However, the interaction effect reveals that the difference in cognitive performance between 6 hours and 8 hours of sleep is significantly greater when combined with high caffeine intake compared to low caffeine intake.