Lecture 4 - Experiment

Question 1

Define the term “experiment”

Answer 1

An observational process characterized by control of the background factors (often through manipulation), intervention on the real target variable through manipulation and observation of the difference caused by this intervention.

Question 2

What is an experiment characterized by?

Answer 2

Activities like observation, manipulation, control etc characterizes experimenting. Often offer justification for accepting or rejecting a hypothesis.

Control of background variables through manipulation
Intervention on target variables through manipulation
Observation of difference produced by intervention

Question 3

Explain Mills Method of Difference

Answer 3

By creating two identical groups, and making an intervention on only one of them, the causal effect of that intervention can be studied, since the intervention is the only factor different between the groups.

Example:
Case A: Group 1 lives in the same area, follows the same diet, and has similar demographics as Group 2. However, Group 1 drank water from a specific well.
Case B: Group 2 lives in the same area, follows the same diet, and has similar demographics as Group 1 but did not drink from that well.
Method of Difference suggests that drinking from the well could be the cause of the illness.

Question 4

What are “control group” and “ treatment group”

Answer 4

Treatment group = a class that is subjected to the intervention
Control group = a contrast class that is not subjected to the intervention, but is similar to the treatment group in all other relevant aspects.

Question 5

Name the 5 “non-experimental practises”

Answer 5

1. Observational studies
2. Natural experiments
3. Field experiments
4. Laboratory experiments
5. Simulation experiment

Question 6

What is observational studie?

Answer 6

An observational process characterized by control of the background factors and observation of an outcome, lacking an intervention

Question 7

What is Natural experiment?

Answer 7

An observational study where circumstances are organized as if there had been manipulation for intervention and control but no manipulation is in fact done.

Question 8

What is field experiment?

Answer 8

An experiment where there is manipulation for intervention, but where several background variables are not controlled, in particular not controlled through manipulation.

Question 9

What is Laboratory experiment?

Answer 9

An experiment where there is manipulation for intervention and where all or most relevant variables are controlled.

Question 10

What is simulation experiment (model studies)?

Answer 10

When you construct a representation of a real system on a computer, and then perform various interventions on that representation.

Question 11

Explain internal and external validity

Answer 11

• Internal Validity: An inference in a scientific study (an experiment, an observational study or a model) is internally valid, if the relation between the studied factor and observed effect inferred from the study is indeed true and is not confounded by uncontrolled background factors.
• External Validity: An inference from an experimental system to a different target of interest is externally valid, if the conclusion holds not only for the system but also for the target, for instance an inference from a sample to a population.

Question 12

What are experimental errors? Name the different types.

Answer 12

Variations, deviations or errors arising from the performance of an experiment which affect the precision, reliability or validity of measurements.

Types of errors:
Failing to control for a relevant factor
Observer Effect
Confirmation Bias
Placebo Effect
Selection Bias

Question 13

What is productive error and non-productive error?

Answer 13

Productive error: An experiment can fail but still be productive.

Non-Productive Failures: When we cannot conclude anything of interest.

Question 14

Explain the error “ Failing to control for a relevant factor”

Answer 14

When you draw the wrong conclusion because you have failed to control an important background factor.

Question 15

Explain Confirmation Bias and Selection bias

Answer 15

Confirmation bias: When individuals favor ingormaiton that donfirms their prewxisting beliefs and hypothesis. The observation is registered incorrectly due to psychological properties of the observer.

Selection Bias: When selection might be influenced by the varying factors that have an influence on the target variable.
A factor influences the selection of the sample, or the division into test and control group, which one did not aim to create.

Question 16

Explain observe effect and placebo effect

Answer 16

Observer effect / influence problem - The act of observation changes what is observed.

Placebo effect - The therapeutic effect that an inert substance or treatment, designed to have no therapeutic value, has on a patient

Question 17

How can you detect experimental errors?

Answer 17

1. Apply all available knowledge when designing the experiment
2. Repitition - investigate previosly performed experiments. Can be done if they can be repeated, reproduced, and replicated.

Repetition - Performing a study again, exactly following the description of the original study.

Reproduction - Repeating a study and obtaining the same result as in the original study.

Replication - Performing a variation of a
study with the intention to achieve the same result, where some aspect of the study has been changed to learn how this aspect influenced the result.

Question 18

Explain strategie for experimental control

Answer 18

1. Treatment/control group
2. Holding things constant (like background factors)
3. Elimination (& blindness) (removing background factors)
4. Separation (Registering the effect of a background variable)

Question 19

Explain the terms blidning, Single blinding / subject blinding, experimenter blidning, double blidning

Answer 19

Blinding - Eliminating observer effect or the influence problem through limiting information about the study to participants or observers.

Single blinding / subject blinding - Blinding the participants in a study, for instance about whether they are in the test or control group.

Experimenter blinding - Blinding the observers in a study, for instance about which subjects are in the test and control group.

Double blinding - A study is double blind if both subject blinding and experimenter
blinding are implemented.

Question 20

What is “Randomization”? Difference from “Random sampling”?

Answer 20

Randomization - Using a random process to divide into test and control, for instance by flipping a coin.

Random sampling - Using a random process to collect a sample from the population. Not to be confused with randomization.

Question 21

What is Randomized Controlled Trials? What are the arguments to use it?

Answer 21

RCT is when no one knows who is in the treatment or who is in the control group.

Arguments:
* Randomization eliminates selection bias.
* Randomization helps convince others that you have not rigged the treatment/control division in favor of the outcome you want.
* Randomization facilitates blinding of the identity of treatments from investigators, participants and assessors.

Question 22

What is the problem with transferring RCT results?

Answer 22

We cannot transfer the results, unless we can argue for that there are the same similarities within the samples, as in the population.

Question 23

Explain Evidential Hierarchies

Answer 23

A ranking of types of evidence based on the processes that produced this evidence.