delkurs 1: Experimentdesign

Question 1

sensitivity

Answer 1

Refers to the likelihood in an experiment that the effect of an independent variable will be detected when that variable does, indeed, have an effect; sensitivity is increased to the extent that error variation is reduced
(e.g., by holding variables constant rather than balancing them).

The ability to detect the effect of the independent variable even if the effect is a small one.

An experiment is more sensitive when there is less variability in participants’ responses within a condition of an experiment = less error variation

Question 2

independent groups design (+/-)

Answer 2

Each separate group of subjects in the experiment

represents a different condition as defined by the level of the independent variable.

Question 3

random group design (+/-)

Answer 3

The most common type of independent groups design in which subjects are randomly assigned to each group such that groups are considered comparable at the start of the experiment.

Question 4

block randomization

Answer 4

The most common technique for carrying out random assignment in the random groups design; each block includes a random order of the conditions, and there are as many blocks as there are subjects in each condition of the experiment.

Block randomization can also be used to order the conditions for each participant in a complete design.

It is effective in balancing practice effects.

In general: the number of blocks in a block-randomized schedule is equal to the number of times each condition is administered, and the size of each block is equal to the number of conditions in the experiment.

Question 5

threats to internal validity

Answer 5

Possible causes of a phenomenon that must be controlled so a clear cause - effect inference can be made.

Question 6

mechanical subject loss

Answer 6

Occurs when a subject fails to complete the experiment because of equipment failure or because of experimenter error.

Question 7

selective subject loss

Answer 7

Occurs when subjects are lost differentially across the conditions of the experiment as the result of some characteristic of each subject that is related to
the outcome of the study.

Question 8

experimenter effects

Answer 8

Experimenters’ expectations that may lead them to treat subjects differently in different groups or to record data in a biased manner.

Question 9

placebo control group (+/-)

Answer 9

Procedure by which a substance that resembles a drug or other active substance but that is actually an inert, or inactive, substance is given to participants.

Acts as a control group ? –> kolla boken

Question 10

double-blind procedure

Answer 10

Both the participant and the observer are kept unaware (blind) of what treatment is being administered.

EX: in an experiment with a placebo control group - neither the observer nor the participant knows who gets the placebo (fake medicin) and who receives the authentic medicin.

Question 11

replication

Answer 11

Repeating the exact procedures used in an experiment to determine whether the same results are obtained.

The most effective way to test the validity of an experiment ? –> kolla boken

Question 12

1. effect size

2. Cohen’s d

Answer 12

1. Index of the strength of the relationship between the independent variable and dependent variable that is independent of sample size.
2. A frequently used measure of effect size in which the difference in means for two conditions is divided by the average variability of participants’ scores (within-group standard deviation).
   Based on Cohen’s guidelines, d values of .20, .50, and .80 represent small, medium, and large effects, of an independent variable.

Question 13

meta-analysis

Answer 13

Analysis of results of several (often, very many) independent experiments investigating the same research area.
The measure used in a meta-analysis is typically effect size.

Question 14

1. Null hypotesis

2. Null hypothesis significance testing (NHST)

Answer 14

1. Assumption used as the first step in statistical inference whereby the independent variable is said to have had no effect.
2. A procedure for statistical inference used to decide whether a variable has produced an effect in a study. NHST begins with the assumption that the variable has no effect (null hypothesis), and probability theory is used to determine the probability that the effect (e.g., a mean difference between conditions) observed in a study would occur simply by error variation (“chance”).

If the likelihood of the observed effect is small (level of significance), assuming the null hypothesis is true, we infer the variable produced a reliable effect (statistically significant).

Question 15

statistically significant

Answer 15

When the probability of an obtained difference in an experiment is smaller than would be expected if error variation alone were assumed to be responsible for the difference, the difference is statistically significant.

Question 16

1. validity
2. internal validity
3. external validity

Answer 16

1. The “truthfulness” of a measure; a valid measure is one that measures what
   it claims to measure.
2. Degree to which differences in performance can be attributed unambiguously to an effect of an independent variable, as opposed to an effect of some other (uncontrolled) variable; an internally valid study is free of confounds.
3. The extent to which the results of a research study can be generalized to different populations, settings, and conditions.

Question 17

t-test (for independent groups)

Answer 17

An inferential test for comparing two means from different groups of subjects.
The test is only used for independent groups = when there are only 2 groups.

Question 18

confidence interval

Answer 18

Indicates the range of values which we can expect to contain a population value with a specified degree of confidence (e.g., 95%).

Question 19

matched groups design (+/-)

Answer 19

Type of independent groups design in which the researcher forms comparable groups by matching subjects on a pretest task and then randomly assigns the members of these matched sets of subjects to the conditions of the experiment.

EX:

Question 20

natural groups design (+/-)

Answer 20

Type of independent groups design in which the conditions represent the selected levels of a naturally occurring independent variable
EX: the individual differences variable age.

Question 21

error variation

Answer 21

The nonsystematic (i.e., random) variation due to differences among subjects within each experiment group.

What you want to know is: how likely is it that the difference you have observed is only due to chance (not to the effect of your independent variable)?

The presence of error variation poses a potential problem because the means of the different groups in the experiment may differ simply because of error variation, not because the independent variable has an effect.

The less error variation, the easier it is to detect the effect of an independent variable

Question 22

Type I error

Answer 22

The probability of rejecting the null hypothesis when it is: -

• true,
• equal to the level of significance
• or alpha

Question 23

Type II error

Answer 23

The probability of failing to reject the null hypothesis when it is false.

Question 24

1. population

2. sample

Answer 24

1. Set of all the cases of interest.
   EX: alla studenter på GU
2. Something less than all the cases of interest; in survey research, a subset of the population actually drawn from the sampling frame.
   EX: frivilliga studenter vid IT-fakulteten

Question 25

level of significance

Answer 25

The probability when testing the null hypothesis that is used to indicate whether an outcome is statistically significant.
Level of significance, or alpha, is equal to the probability of a Type I error.

Question 26

confounding

Answer 26

Occurs when the independent variable of interest systematically covaries with a second, unintended independent variable.

Question 27

individual differences variable

or subject variable

Answer 27

A characteristic or trait that varies consistently across
individuals, such as level of depression, age, intelligence, gender.

Because this variable is formed from preexisting groups (i.e., it occurs “naturally”) an individual differences variable is sometimes called a natural groups variable.

Question 28

independent variable

Answer 28

Factor for which the researcher manipulates at least two levels in order to determine its effect on behavior.

Question 29

F-test

Answer 29

In the analysis of variance, or ANOVA, the ratio of between-group variation and within-group or error variation.

Is used when there are more than 2 groups.

Question 30

dependent variable

Answer 30

Measure of behavior used by the researcher to assess the effect (if any) of the independent variable.

Question 31

demand characteristics

Answer 31

Cues and other information used by participants to guide their behavior in a psychological study, often leading participants to do what they believe the observer (experimenter) expects them to do.

Question 32

comparison of two means

Answer 32

A statistical technique that can be applied (usually after obtaining a statistically significant omnibus F-test) to locate the specific source of systematic variation in an experiment by comparing means two at a time.

Question 33

ANOVA

Answer 33

The analysis of variance, or ANOVA, is the most commonly used inferential test for examining a null hypothesis when comparing more than two means in a single-factor study, or in studies with more than one factor (i.e., independent variable). The ANOVA test is based on analyzing different sources of variation in an experiment.

Question 34

repeated measure design

Answer 34

+ there can be no confounding by individual differences variables

Threats:
- participants may change over time

Is used when:
- the experimental procedure requires that participants compare two or more stimuli relative to one another

- when the research question involves studying changes in participants’ behavior over time, such as in a learning experiment    - research areas such as psychophysics or in scaling

Question 35

practice effects

Answer 35

The changes participants undergo with repeated testing in the repeated measures designs.

• get better at the task (thanks to the repeated practice)
• get worse at the task because of boredom or fatigue

In general, practice effects should be balanced across the conditions in repeated measures designs so that practice effects “average out” across con- ditions

Question 36

1. counterbalancing

a) complete design
b) incomplete design

Answer 36

1. The specific techniques for balancing practice effects
   a) In the complete design, practice effects are balanced for each participant by administering the conditions to each participant several times, using different orders each time. Each participant can thus be considered a “complete” experiment.
   b) In the incomplete design, each condition is administered to each participant only once. The order of administering the conditions is varied across participants rather than for each participant, as is the case in the complete design. Practice effects in the incomplete design average out when the results are combined for all participants.

Question 37

ABBA counterbalancing

Answer 37

ABBA counterbalancing can be used to balance practice effects in the complete design with as few as two administrations of each condition.

Usually this technique is used when the number of conditions and the number of repetitions of each condition are relatively small, tho it can be used effectively with larger numbers of repetitions of the cycle as well.

This technique is used to balance practice effects when it is not possible to administer each condition often enough for the averaging process of block randomization to work effectively.

ABBA counterbalancing is appropriately used only when practice effects are linear, i.e. when the same amount of practice effects is added to or subtracted from performance on each successive trial.

Question 38

anticipation effects

Answer 38

occur when a participant develops expectations about which condition should occur next in the sequence.

The participant’s response to that condition may then be influenced more by this expectation than by the actual experience of the condition itself.

(When anticipation effects are likely, block randomization should be used rather than ABBA counterbalancing in order to balance practice effects)

Question 39

N! (which is read “N factorial”)

Answer 39

In general, there are N! possible orders with N conditions,
where N! equals N(N - 1) (N - 2) . . . (N - [N - 1])

EX: there are six possible orders with three conditions: which is 3! (3 x 2 x 1 = 6).

Question 40

1. the mean

2. standard deviation

Answer 40

The two most common descriptive statistics used to summarize the results of experiment

1. = medelvärde
   - a measure of central tendency (average)
2. represented by sigma σ
   - a measure of variability
   - a measure of spread: how spread out is the data/how closely is the data clustered around the mean(average)?
   low deviation –> the data is close to mean/average
   high deviation –> the data is spread over a wider range of values

Standard deviation is usually used to understand wether a point of data is standard and expected or if is unusual and unexpected.

Question 41

differential transfer

Answer 41

A serious potential problem that can arise in repeated measures designs. It arises when performance in one condition differs depending on the condition that precedes it.

It also tends to underestimate differences between the conditions and thereby reduces the external validity of the findings. Therefore, when differential transfer could occur, researchers should choose an independent groups design.