Experiments Flashcards
Mill’s method of difference involves setting up two separate systems S1 and S2. What is true about these systems? Mark all correct options.
a) Any background factors that are controlled in S1 must also be controlled in S2.
b) If we do an intervention in S1, then we only need to observe the result in S1, not in S2.
c) Using two systems enables us to exclude alternative causes of the observed results.
d) Any intervention done on S1 must also be done on S2.
Answer: a), c)
Which of the following statements about Mill’s Method of Difference are true? Mark all correct options.
a) Mill’s Method of Difference exhibits the reasoning behind testing hypotheses with experiments.
b) Mill’s Method of Difference elucidates the ways in which an experimental process can go wrong.
c) The main question that Mill’s Method of difference asks is: what causes event E?
d) Although it is an illustrative thought experiment, Mill’s Method of Difference has little to do with how actual experiments are conducted.
Answer: a), b), c)
Which of the following empirical practices constitute experiments? Mark all correct options.
a) Measuring the boiling point of a liquid in a pressure chamber to observe how this changes when you change the pressure.
b) Measuring the temperature of arctic lakes by taking samples at different depths.
c) Observing the effects of a natural catastrophe on a local economy.
d) Tracking the orbits of two of Saturn’s moons to see if they’re affected by each other’s gravitational pull.
e) Testing a medical treatment for potential side-effects by giving it to a number of patients and observing their health.
Answer: a), e)
Imagine the following scenario.
You are the head researcher at AI Robotics Inc. In six months’ time you are going to release a new line of humanoid robots. You want to know how well they are going to keep their balance doing regular household tasks such as picking up the newspaper, cleaning up, and similar things. To get some data to answer your question, you build a computer program that constructs a virtual 3D environment in the form of a house, and impose artificial gravity onto the environment. You then import the schematics for the new humanoid robot into this environment and then watch it perform such household tasks in the virtual world.
Did you just perform an experiment? Mark all correct options.
a) No. Simulations are not experiments because they do not manipulate the real variables.
b) Yes. We controlled the background variables, manipulated our target variable and observed or recorded the outcome.
c) Yes. Experiments are mostly about predicting what will happen, and this simulation gives us information to base our judgement on.
d) No. We constructed an observational study.
Answer: a)
Which of the following, broadly construed alternatives describe general types of error in experiments? Mark all correct options.
a) A failure to intervene on the target variable.
b) A failure to correctly observe the outcome of the experiment.
c) A failure to reconcile the hypothesis with the common sense conception of the investigated phenomenon.
d) A failure to implement control over background variables.
Answer: a), b), d)
General Feedback
The false option(s) might point you to problems which, although authentic, are not about errors within the experiment.
Imagine that you are performing an experiment on the heat emission of a certain type of exhaust engine. Your hypothesis is that a new kind of material used in the engine will result in a more even heat emission. You set up a prototype of the engine in a laboratory environment and carefully plan every step of the experimental procedure, including what equipment you will use to control background factors and measure the relevant parameters. You then perform the experiment according to your plan and note the results, which support your hypothesis.
What does it take for your experiment to be repeatable? Mark all correct options.
a) A separate research team must test the same hypothesis and obtain the same result.
b) None of the other alternatives are required. If all steps specified in the question text are taken then, by definition, the experiment is repeatable.
c) The results as well as the mentioned details about experimental setup, measuring equipment etc. must be published or made available in some way, so that other researchers can take part of the information.
d) Your own laboratory and equipment must be made available to future researchers.
Answer: c)
“Repeatable” means the original researcher can get consistent results by repeating their own work.
“Reproducible” means another researcher can get consistent or similar results by closely following the original methods.
“Replicable” means a different researcher can get consistent or similar results using potentially different methods, equipment, or conditions.
Imagine the following scenario.
You have been tasked to investigate whether Bright Light Therapy affects people’s self-assessments of their level of well-being. You decide to conduct an experiment as a way of providing an answer to this question. A sample is randomly generated from a population of people who’ve agreed to participate in the study. You are now to decide which control measures to implement in your investigation.
Which of the following options are relevant measures for establishing control? Mark all correct options.
a) You repeat the experiment in different times of the year.
b) You influence the assessments people make of their level of well-being when answering a standardized questionnaire.
c) You identify the participants randomly chosen and select only the ones you think will lead to the experimental result you are expecting.
d) You expose only a part of the group to the bright light therapy and only pretend to do so for the remaining people.
e) You make sure that the population from which the sample is taken is not skewed in some sense.
Answer: a), d), e)
In some experimental cases, it’s not possible to eliminate all background factors that (might) have a disturbing effect on the outcome. In such a situation, is it necessarily the case that using a model simulation is a preferrable method of investigation? Mark the correct option.
a) No. Disturbing factors cannot always be eliminated. While a preferred control measure in many instances, sometimes it suffices to register different effects separately and take them into account in the analysis of the data.
b) Yes. Disturbing factors always influence tests in ways that must be controlled for through elimination.
c) Yes. Models can provide us with the data required to answer questions better than an experiment can in almost all instances.
d) No. It is always possible for us to, in some more or less abstract way, eliminate disturbing background factors. Therefore using a model is not a preferred method of investigation.
Answer: a)
Imagine the following scenario.
You want to test how sensitive a set of plants are to a certain chemical compound. First, you devise a way of safely subjecting such plants to this compound. Second, you prepare yourself to record the levels of certain toxins in the plants throughout a period of two weeks. Before all of this, you wish to separate the plants into a treatment and a control group.
Which methods below can be used for dividing the plants into control and treatment group? Mark all correct options.
a) Use a lottery wheel to distribute the items based on whether they land on a red or blue area (of which there are equally many) on the wheel, and repeating this process until known disturbing factors only occur in the control group.
b) Flipping a fair coin and distributing the groups based on which side it landed, and check that known relevant factors are indeed equally distributed in treatment and control group.
c) Forming a treatment group of the plants that seem healthiest according to some known parameters and putting the rest in the control group.
d) Use a random number generator and distribute items to the two groups based on whether the number was even or odd, and check that known relevant factors are indeed equally distributed in treatment and control group.
Answer: b), d)
What are some purposes for experimenters’ choice of using the method of randomization? Mark all correct options.
a) Randomization facilitates blinding.
b) Randomization ensures that all background factors are evenly distributed throughout the groups.
c) Randomization helps to establish the selection based on one singular factor.
d) Randomization eliminates the bias of consciously or unconsciously choosing particular items from the sample.
e) Randomization helps convince others that you have not rigged the treatment/control division in favor of the outcome you want
Answer: a), d), e)
What is true about external validity? Mark all correct options.
a) To ensure external validity we need our sample to be representative of the population.
b) Randomly assigning our sample into control and treatment group is a way to ensure external validity.
c) The results of an RCT (randomized controlled trial) will by design be externally valid.
d) Randomly choosing a sufficiently large sample from the population is a way to ensure external validity.
Answer: a), d)
